2026-05-11T14:27:48,881 Created temporary directory: /tmp/pip-ephem-wheel-cache-9hhgi13n 2026-05-11T14:27:48,883 Created temporary directory: /tmp/pip-build-tracker-ro2flxcj 2026-05-11T14:27:48,883 Initialized build tracking at /tmp/pip-build-tracker-ro2flxcj 2026-05-11T14:27:48,884 Created build tracker: /tmp/pip-build-tracker-ro2flxcj 2026-05-11T14:27:48,884 Entered build tracker: /tmp/pip-build-tracker-ro2flxcj 2026-05-11T14:27:48,885 Created temporary directory: /tmp/pip-wheel-9ffhwy8y 2026-05-11T14:27:48,888 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2026-05-11T14:27:48,891 Created temporary directory: /tmp/pip-ephem-wheel-cache-gfygn71x 2026-05-11T14:27:48,916 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2026-05-11T14:27:48,919 2 location(s) to search for versions of lm-eval: 2026-05-11T14:27:48,919 * https://pypi.org/simple/lm-eval/ 2026-05-11T14:27:48,919 * https://www.piwheels.org/simple/lm-eval/ 2026-05-11T14:27:48,920 Fetching project page and analyzing links: https://pypi.org/simple/lm-eval/ 2026-05-11T14:27:48,921 Getting page https://pypi.org/simple/lm-eval/ 2026-05-11T14:27:48,922 Found index url https://pypi.org/simple 2026-05-11T14:27:49,075 Fetched page https://pypi.org/simple/lm-eval/ as application/vnd.pypi.simple.v1+json 2026-05-11T14:27:49,082 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/1b/5f/7841febb99c12ffb453d33a67b9841e89dba18c388b644bf22b81d137fc4/lm_eval-0.0.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T14:27:49,083 Found link https://files.pythonhosted.org/packages/21/5a/feb5ff3a1591ca963c54873d39116b0e6a4f80e493e961ac08569709c5d7/lm_eval-0.0.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.0.1 2026-05-11T14:27:49,084 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/f3/a7/63cbce8b51de25fabb1c49f3a3fd1704faaacadb5ed816401f800e4d2dbd/lm_eval-0.2.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T14:27:49,085 Found link https://files.pythonhosted.org/packages/c5/fd/edd21b0f258b4ec0260f99f5b2ac3864f7cddc8fb7c83bbb2379a6aab975/lm_eval-0.2.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.2.0 2026-05-11T14:27:49,086 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/61/c5/bff92e6b61fc2b0c1b7ac769633731910152e5176a404912ce7c07329ba0/lm_eval-0.3.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T14:27:49,087 Found link https://files.pythonhosted.org/packages/c4/f8/58abc65390a758c8c2e5f1d8bb9b58d7885d02535d5f48de27006453d07e/lm_eval-0.3.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.3.0 2026-05-11T14:27:49,088 Found link https://files.pythonhosted.org/packages/45/e0/05001c2e56e2f8f793189442176432ddb89a2f1f1ef00ea154d0bc00fe37/lm_eval-0.4.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.0 2026-05-11T14:27:49,089 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/49/2d/39f7a25ab663cb45cfc7773b85980f01df44853cc427d00dce94c90b43e6/lm_eval-0.4.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,090 Found link https://files.pythonhosted.org/packages/5a/02/1c7f1ac2f139f4c05af5b94e2c4f88a70404fa0b0c22a5fb04dec0216b03/lm_eval-0.4.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.1 2026-05-11T14:27:49,090 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/0c/5a/64cf703b62ac7ada09a514c16c7136bb4ea7ef3030eb4c3d780900a5a634/lm_eval-0.4.2-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,091 Found link https://files.pythonhosted.org/packages/75/ca/18814743ba3b42d19d8524a4f771e9c3a7aa02cd4c579747a0f513907205/lm_eval-0.4.2.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.2 2026-05-11T14:27:49,092 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/39/2b/eb15fb57ba48b3cebfd302713d680d9917101867e2b5fbfd266e88b89ff3/lm_eval-0.4.3-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,093 Found link https://files.pythonhosted.org/packages/2e/69/dbff28f17e17521735460b25645479e85b58d8ad7ea8488e4672ab52b58b/lm_eval-0.4.3.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.3 2026-05-11T14:27:49,094 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/ad/e1/ac8d7d26b4d6a9eea8859d438691b5d8209a37be2117b3e61f247de2d9e7/lm_eval-0.4.4-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,095 Found link https://files.pythonhosted.org/packages/c6/a8/4dcffd9ad0ddb6c479f93d8316a108a6f9b4d81756c892f150b6c307e8d4/lm_eval-0.4.4.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.4 2026-05-11T14:27:49,095 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/c1/78/fb3ea82c77ccc4dbbd6cafceb15ca01c1cd4d778217d572d65c919b0643f/lm_eval-0.4.5-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,096 Found link https://files.pythonhosted.org/packages/68/7a/cb1993a8d217a5b4c6e52f612a677c4f02c3412d2e3093ecbf6f6413136c/lm_eval-0.4.5.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.5 2026-05-11T14:27:49,097 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/45/b9/1b4e3268b590d9ff16e087685d9526455bb677c3e4d0caeba4451f20c586/lm_eval-0.4.7-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,098 Found link https://files.pythonhosted.org/packages/53/c9/b5d03d5b2bf6819008e377844999fbd04ab00dff0c43728957f1c90a53c5/lm_eval-0.4.7.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.7 2026-05-11T14:27:49,098 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/c3/0b/36d6117f644f3685e6b87005ecd7051d01e9cdcf617e8e671102c1546de2/lm_eval-0.4.8-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T14:27:49,100 Found link https://files.pythonhosted.org/packages/ba/b2/fe23d318c5f645636f9f776c373d1ba68aa55c998894660613a09b0fe2ec/lm_eval-0.4.8.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9), version: 0.4.8 2026-05-11T14:27:49,100 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/b6/4f/3ab58de3dfee26f693d149824d9d62723f771392c4756becccbcbaaa4e03/lm_eval-0.4.9-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T14:27:49,101 Found link https://files.pythonhosted.org/packages/e5/ab/866d8957cddbb26510e5cf5f7c5ec01e179f82d54d2dafd3a6c5c31fc200/lm_eval-0.4.9.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9), version: 0.4.9 2026-05-11T14:27:49,102 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/18/ba/3a992d435deaf5461f50417bd7ba38a21831a04520941a9ecf1020dcf900/lm_eval-0.4.9.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T14:27:49,103 Found link https://files.pythonhosted.org/packages/b9/81/943efd4d008080625ddb135d961fc40e78fa75c61f4ce1b4b1244c667571/lm_eval-0.4.9.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9), version: 0.4.9.1 2026-05-11T14:27:49,103 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/6c/50/3903694e69c94ae8dcec654843cb99799e45137e0d1a0a581009aa592673/lm_eval-0.4.9.2-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T14:27:49,105 Found link https://files.pythonhosted.org/packages/e1/5e/e6d70f4aa2acd7932a873c51de93ddf2dbd1c43aa6f6c86314ab3c0d279d/lm_eval-0.4.9.2.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.9.2 2026-05-11T14:27:49,105 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/23/5b/d0762a01f140e945aa7a44c92bb0c5c149c2c2917278c02eef6f5053b8e1/lm_eval-0.4.10-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T14:27:49,106 Found link https://files.pythonhosted.org/packages/7c/1c/858cc03ac7960bc7a1323d40a0d912ebef560825075504238f6721ad8a3a/lm_eval-0.4.10.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.10 2026-05-11T14:27:49,106 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/df/b3/8c2923ad4c4d911307e20ab9c7d7869d3811b5a76fe6dbe53678aafbeb04/lm_eval-0.4.11-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T14:27:49,107 Found link https://files.pythonhosted.org/packages/18/40/ad60ae17f97902ba4ee67914a94c159552f8643fb4598d1d2430a571f0a2/lm_eval-0.4.11.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.11 2026-05-11T14:27:49,108 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/bd/43/944724465ed73ec2889f58007c114ce8dd2023324bc86d3e6f41ce00e2e8/lm_eval-0.4.12-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T14:27:49,109 Found link https://files.pythonhosted.org/packages/12/32/783623b1d1fe7c89ebd774db46bef4c787fe93e7c5e1b4877c7dfbf1abbe/lm_eval-0.4.12.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.12 2026-05-11T14:27:49,110 Fetching project page and analyzing links: https://www.piwheels.org/simple/lm-eval/ 2026-05-11T14:27:49,110 Getting page https://www.piwheels.org/simple/lm-eval/ 2026-05-11T14:27:49,111 Found index url https://www.piwheels.org/simple 2026-05-11T14:27:49,303 Fetched page https://www.piwheels.org/simple/lm-eval/ as text/html 2026-05-11T14:27:49,310 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.11-py3-none-any.whl#sha256=7a994cec7f884dff2892c895be7943f2e5daf899d1a0c96862a09e86d38317ef (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T14:27:49,311 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.10-py3-none-any.whl#sha256=f974c8439950d7e007710fae43c1ff085ba2724b8c9c29456987c442a14ddc4a (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T14:27:49,311 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.9.2-py3-none-any.whl#sha256=6433acae848f5040a05aec1f8b4d946c1874207195592064a0dccbf11e6a3bf7 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T14:27:49,312 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.9.1-py3-none-any.whl#sha256=97ecff896cf8c8a3461730701532cb297638f594631d30e0a0e64bedacc771ce (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T14:27:49,312 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.9-py3-none-any.whl#sha256=aae7521fd8819d6dd3ae674c07816f449a70669bef1db6cdce2968c21ad1f7af (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T14:27:49,313 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.8-py3-none-any.whl#sha256=1ed005366d8f0154fa200db6e0eb56cc5ef33df2bbd2705d19c742e4283593c8 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T14:27:49,314 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.7-py3-none-any.whl#sha256=3b462a73021c5411e6b4491d07d5c4c009843435a2c9350ae5949fbf4a196ed9 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,314 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.5-py3-none-any.whl#sha256=c1292031153f79a22db4193ed91074a1c57ffaec3d9768215a455efd53ff5f02 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,315 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.4-py3-none-any.whl#sha256=1a6e6b514b470ad4c795439aa4cf9cb88b1360d915c7b3108ae5b47e3f7b36fc (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,315 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.3-py3-none-any.whl#sha256=7f4cb7cbd7ea20f2624f1315e69dfbc884cc187749c85760f9b4ec03e7efe491 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,316 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.2-py3-none-any.whl#sha256=a0bcaff492a09947d8a2b80400f224113b1e313da6a688084711130da18e1af7 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,316 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.1-py3-none-any.whl#sha256=796615edf52f5673187a884a9d77c7d755fe95ca2d16fe77a1f042730eb957b3 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,317 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.0-py3-none-any.whl#sha256=1a863b5f478f2e66e921dbbf2e4dcda06a923021763f01b59c5b03bdd3af01c2 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T14:27:49,318 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.3.0-py3-none-any.whl#sha256=498b8b8954c1f9c17f46e3ec096e9be6b9c96ee70560ee613a4eb9c7b9d31644 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T14:27:49,318 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.2.0-py3-none-any.whl#sha256=e06d3d7b6016be832e6889cbc9c4787b99156ba57f8feb31d3aeb27304c6558c (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T14:27:49,319 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.0.1-py3-none-any.whl#sha256=0afc289f69286f71017fb9811dfea6cda7c703bf693ad43106fbbff1f164cf14 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T14:27:49,319 Skipping link: not a file: https://www.piwheels.org/simple/lm-eval/ 2026-05-11T14:27:49,320 Skipping link: not a file: https://pypi.org/simple/lm-eval/ 2026-05-11T14:27:49,342 Given no hashes to check 1 links for project 'lm-eval': discarding no candidates 2026-05-11T14:27:49,362 Collecting lm-eval==0.4.12 2026-05-11T14:27:49,364 Created temporary directory: /tmp/pip-unpack-d63zshp8 2026-05-11T14:27:49,594 Downloading lm_eval-0.4.12.tar.gz (3.4 MB) 2026-05-11T14:28:13,290 Added lm-eval==0.4.12 from https://files.pythonhosted.org/packages/12/32/783623b1d1fe7c89ebd774db46bef4c787fe93e7c5e1b4877c7dfbf1abbe/lm_eval-0.4.12.tar.gz to build tracker '/tmp/pip-build-tracker-ro2flxcj' 2026-05-11T14:28:13,298 Created temporary directory: /tmp/pip-build-env-1oizwjbl 2026-05-11T14:28:13,303 Installing build dependencies: started 2026-05-11T14:28:13,304 Running command pip subprocess to install build dependencies 2026-05-11T14:28:14,477 Using pip 23.0.1 from /usr/lib/python3/dist-packages/pip (python 3.11) 2026-05-11T14:28:14,909 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2026-05-11T14:28:14,931 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2026-05-11T14:28:16,705 Collecting setuptools>=64.0 2026-05-11T14:28:16,778 Using cached https://www.piwheels.org/simple/setuptools/setuptools-82.0.1-py3-none-any.whl (1.0 MB) 2026-05-11T14:28:19,730 Installing collected packages: setuptools 2026-05-11T14:28:22,904 Successfully installed setuptools-82.0.1 2026-05-11T14:28:23,169 Installing build dependencies: finished with status 'done' 2026-05-11T14:28:23,175 Getting requirements to build wheel: started 2026-05-11T14:28:23,177 Running command Getting requirements to build wheel 2026-05-11T14:28:24,419 /tmp/pip-build-env-1oizwjbl/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2026-05-11T14:28:24,420 !! 2026-05-11T14:28:24,421 ******************************************************************************** 2026-05-11T14:28:24,421 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2026-05-11T14:28:24,422 By 2027-Feb-18, you need to update your project and remove deprecated calls 2026-05-11T14:28:24,423 or your builds will no longer be supported. 2026-05-11T14:28:24,424 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2026-05-11T14:28:24,424 ******************************************************************************** 2026-05-11T14:28:24,425 !! 2026-05-11T14:28:24,426 corresp(dist, value, root_dir) 2026-05-11T14:28:24,513 running egg_info 2026-05-11T14:28:24,519 writing lm_eval.egg-info/PKG-INFO 2026-05-11T14:28:24,543 writing dependency_links to lm_eval.egg-info/dependency_links.txt 2026-05-11T14:28:24,544 writing entry points to lm_eval.egg-info/entry_points.txt 2026-05-11T14:28:24,556 writing requirements to lm_eval.egg-info/requires.txt 2026-05-11T14:28:24,558 writing top-level names to lm_eval.egg-info/top_level.txt 2026-05-11T14:28:27,765 reading manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T14:28:28,522 reading manifest template 'MANIFEST.in' 2026-05-11T14:28:28,522 warning: manifest_maker: MANIFEST.in, line 1: 'recursive-include' expects ... 2026-05-11T14:28:28,524 adding license file 'LICENSE.md' 2026-05-11T14:28:29,344 writing manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T14:28:29,471 Getting requirements to build wheel: finished with status 'done' 2026-05-11T14:28:29,474 Created temporary directory: /tmp/pip-modern-metadata-hmzk0k2q 2026-05-11T14:28:29,477 Preparing metadata (pyproject.toml): started 2026-05-11T14:28:29,478 Running command Preparing metadata (pyproject.toml) 2026-05-11T14:28:30,342 /tmp/pip-build-env-1oizwjbl/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2026-05-11T14:28:30,342 !! 2026-05-11T14:28:30,344 ******************************************************************************** 2026-05-11T14:28:30,344 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2026-05-11T14:28:30,345 By 2027-Feb-18, you need to update your project and remove deprecated calls 2026-05-11T14:28:30,346 or your builds will no longer be supported. 2026-05-11T14:28:30,348 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2026-05-11T14:28:30,348 ******************************************************************************** 2026-05-11T14:28:30,349 !! 2026-05-11T14:28:30,349 corresp(dist, value, root_dir) 2026-05-11T14:28:30,442 running dist_info 2026-05-11T14:28:30,452 creating /tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info 2026-05-11T14:28:30,453 writing /tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/PKG-INFO 2026-05-11T14:28:30,476 writing dependency_links to /tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/dependency_links.txt 2026-05-11T14:28:30,477 writing entry points to /tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/entry_points.txt 2026-05-11T14:28:30,489 writing requirements to /tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/requires.txt 2026-05-11T14:28:30,490 writing top-level names to /tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/top_level.txt 2026-05-11T14:28:30,492 writing manifest file '/tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/SOURCES.txt' 2026-05-11T14:28:35,394 reading manifest file '/tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/SOURCES.txt' 2026-05-11T14:28:35,395 reading manifest template 'MANIFEST.in' 2026-05-11T14:28:35,396 warning: manifest_maker: MANIFEST.in, line 1: 'recursive-include' expects ... 2026-05-11T14:28:35,397 adding license file 'LICENSE.md' 2026-05-11T14:28:35,906 writing manifest file '/tmp/pip-modern-metadata-hmzk0k2q/lm_eval.egg-info/SOURCES.txt' 2026-05-11T14:28:36,010 creating '/tmp/pip-modern-metadata-hmzk0k2q/lm_eval-0.4.12.dist-info' 2026-05-11T14:28:36,169 Preparing metadata (pyproject.toml): finished with status 'done' 2026-05-11T14:28:36,178 Source in /tmp/pip-wheel-9ffhwy8y/lm-eval_3f778cbaa3dc468497ad567d0d1688ba has version 0.4.12, which satisfies requirement lm-eval==0.4.12 from https://files.pythonhosted.org/packages/12/32/783623b1d1fe7c89ebd774db46bef4c787fe93e7c5e1b4877c7dfbf1abbe/lm_eval-0.4.12.tar.gz 2026-05-11T14:28:36,179 Removed lm-eval==0.4.12 from https://files.pythonhosted.org/packages/12/32/783623b1d1fe7c89ebd774db46bef4c787fe93e7c5e1b4877c7dfbf1abbe/lm_eval-0.4.12.tar.gz from build tracker '/tmp/pip-build-tracker-ro2flxcj' 2026-05-11T14:28:36,193 Created temporary directory: /tmp/pip-unpack-qq1pznhb 2026-05-11T14:28:36,194 Building wheels for collected packages: lm-eval 2026-05-11T14:28:36,198 Created temporary directory: /tmp/pip-wheel-ek8ksrwg 2026-05-11T14:28:36,199 Destination directory: /tmp/pip-wheel-ek8ksrwg 2026-05-11T14:28:36,201 Building wheel for lm-eval (pyproject.toml): started 2026-05-11T14:28:36,202 Running command Building wheel for lm-eval (pyproject.toml) 2026-05-11T14:28:37,004 /tmp/pip-build-env-1oizwjbl/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2026-05-11T14:28:37,004 !! 2026-05-11T14:28:37,005 ******************************************************************************** 2026-05-11T14:28:37,006 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2026-05-11T14:28:37,007 By 2027-Feb-18, you need to update your project and remove deprecated calls 2026-05-11T14:28:37,008 or your builds will no longer be supported. 2026-05-11T14:28:37,009 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2026-05-11T14:28:37,009 ******************************************************************************** 2026-05-11T14:28:37,010 !! 2026-05-11T14:28:37,011 corresp(dist, value, root_dir) 2026-05-11T14:28:37,094 running bdist_wheel 2026-05-11T14:28:37,111 running build 2026-05-11T14:28:37,112 running build_py 2026-05-11T14:28:37,118 creating build/lib/lm_eval 2026-05-11T14:28:37,120 copying lm_eval/__main__.py -> build/lib/lm_eval 2026-05-11T14:28:37,122 copying lm_eval/evaluator.py -> build/lib/lm_eval 2026-05-11T14:28:37,124 copying lm_eval/result_schema.py -> build/lib/lm_eval 2026-05-11T14:28:37,127 copying lm_eval/__init__.py -> build/lib/lm_eval 2026-05-11T14:28:37,129 copying lm_eval/utils.py -> build/lib/lm_eval 2026-05-11T14:28:37,131 copying lm_eval/evaluator_utils.py -> build/lib/lm_eval 2026-05-11T14:28:37,134 copying lm_eval/defaults.py -> build/lib/lm_eval 2026-05-11T14:28:37,136 creating build/lib/lm_eval/caching 2026-05-11T14:28:37,137 copying lm_eval/caching/__init__.py -> build/lib/lm_eval/caching 2026-05-11T14:28:37,139 copying lm_eval/caching/cache.py -> build/lib/lm_eval/caching 2026-05-11T14:28:37,141 creating build/lib/lm_eval/api 2026-05-11T14:28:37,142 copying lm_eval/api/task.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,145 copying lm_eval/api/model.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,148 copying lm_eval/api/metrics.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,151 copying lm_eval/api/instance.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,152 copying lm_eval/api/registry.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,155 copying lm_eval/api/samplers.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,157 copying lm_eval/api/__init__.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,158 copying lm_eval/api/utils.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,160 copying lm_eval/api/filter.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,162 copying lm_eval/api/group.py -> build/lib/lm_eval/api 2026-05-11T14:28:37,165 creating build/lib/lm_eval/decontamination 2026-05-11T14:28:37,166 copying lm_eval/decontamination/__init__.py -> build/lib/lm_eval/decontamination 2026-05-11T14:28:37,167 copying lm_eval/decontamination/archiver.py -> build/lib/lm_eval/decontamination 2026-05-11T14:28:37,169 copying lm_eval/decontamination/decontaminate.py -> build/lib/lm_eval/decontamination 2026-05-11T14:28:37,172 copying lm_eval/decontamination/janitor.py -> build/lib/lm_eval/decontamination 2026-05-11T14:28:37,174 creating build/lib/lm_eval/prompts 2026-05-11T14:28:37,175 copying lm_eval/prompts/__init__.py -> build/lib/lm_eval/prompts 2026-05-11T14:28:37,178 creating build/lib/lm_eval/tasks 2026-05-11T14:28:37,179 copying lm_eval/tasks/manager.py -> build/lib/lm_eval/tasks 2026-05-11T14:28:37,182 copying lm_eval/tasks/_yaml_loader.py -> build/lib/lm_eval/tasks 2026-05-11T14:28:37,184 copying lm_eval/tasks/_index.py -> build/lib/lm_eval/tasks 2026-05-11T14:28:37,187 copying lm_eval/tasks/_factory.py -> build/lib/lm_eval/tasks 2026-05-11T14:28:37,189 copying lm_eval/tasks/__init__.py -> build/lib/lm_eval/tasks 2026-05-11T14:28:37,192 creating build/lib/lm_eval/filters 2026-05-11T14:28:37,192 copying lm_eval/filters/custom.py -> build/lib/lm_eval/filters 2026-05-11T14:28:37,194 copying lm_eval/filters/transformation.py -> build/lib/lm_eval/filters 2026-05-11T14:28:37,196 copying lm_eval/filters/extraction.py -> build/lib/lm_eval/filters 2026-05-11T14:28:37,199 copying lm_eval/filters/selection.py -> build/lib/lm_eval/filters 2026-05-11T14:28:37,200 copying lm_eval/filters/__init__.py -> build/lib/lm_eval/filters 2026-05-11T14:28:37,202 copying lm_eval/filters/decontamination.py -> build/lib/lm_eval/filters 2026-05-11T14:28:37,204 creating build/lib/lm_eval/config 2026-05-11T14:28:37,205 copying lm_eval/config/task.py -> build/lib/lm_eval/config 2026-05-11T14:28:37,208 copying lm_eval/config/evaluate_config.py -> build/lib/lm_eval/config 2026-05-11T14:28:37,210 copying lm_eval/config/__init__.py -> build/lib/lm_eval/config 2026-05-11T14:28:37,212 copying lm_eval/config/group.py -> build/lib/lm_eval/config 2026-05-11T14:28:37,216 creating build/lib/lm_eval/models 2026-05-11T14:28:37,217 copying lm_eval/models/openai_completions.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,220 copying lm_eval/models/megatron_lm.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,224 copying lm_eval/models/sglang_causallms.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,227 copying lm_eval/models/huggingface.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,231 copying lm_eval/models/textsynth.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,233 copying lm_eval/models/utils_hf.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,235 copying lm_eval/models/optimum_habana.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,238 copying lm_eval/models/hf_audiolm.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,240 copying lm_eval/models/hf_vlms.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,243 copying lm_eval/models/neuron_optimum.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,245 copying lm_eval/models/litellm_llms.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,247 copying lm_eval/models/optimum_lm.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,249 copying lm_eval/models/anthropic_llms.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,252 copying lm_eval/models/sglang_generate_API.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,254 copying lm_eval/models/api_models.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,259 copying lm_eval/models/vllm_vlms.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,264 copying lm_eval/models/ibm_watsonx_ai.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,266 copying lm_eval/models/mistral3.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,268 copying lm_eval/models/mamba_lm.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,270 copying lm_eval/models/vllm_causallms.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,273 copying lm_eval/models/__init__.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,275 copying lm_eval/models/utils.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,278 copying lm_eval/models/gguf.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,280 copying lm_eval/models/dummy.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,282 copying lm_eval/models/nemo_lm.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,285 copying lm_eval/models/winml.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,287 copying lm_eval/models/hf_steered.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,290 copying lm_eval/models/trtllm_causallms.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,292 copying lm_eval/models/optimum_ipex.py -> build/lib/lm_eval/models 2026-05-11T14:28:37,295 creating build/lib/lm_eval/loggers 2026-05-11T14:28:37,295 copying lm_eval/loggers/evaluation_tracker.py -> build/lib/lm_eval/loggers 2026-05-11T14:28:37,299 copying lm_eval/loggers/trackio_logger.py -> build/lib/lm_eval/loggers 2026-05-11T14:28:37,301 copying lm_eval/loggers/__init__.py -> build/lib/lm_eval/loggers 2026-05-11T14:28:37,303 copying lm_eval/loggers/utils.py -> build/lib/lm_eval/loggers 2026-05-11T14:28:37,305 copying lm_eval/loggers/wandb_logger.py -> build/lib/lm_eval/loggers 2026-05-11T14:28:37,308 creating build/lib/lm_eval/_cli 2026-05-11T14:28:37,309 copying lm_eval/_cli/validate.py -> build/lib/lm_eval/_cli 2026-05-11T14:28:37,311 copying lm_eval/_cli/ls.py -> build/lib/lm_eval/_cli 2026-05-11T14:28:37,313 copying lm_eval/_cli/__init__.py -> build/lib/lm_eval/_cli 2026-05-11T14:28:37,315 copying lm_eval/_cli/utils.py -> build/lib/lm_eval/_cli 2026-05-11T14:28:37,317 copying lm_eval/_cli/harness.py -> build/lib/lm_eval/_cli 2026-05-11T14:28:37,319 copying lm_eval/_cli/run.py -> build/lib/lm_eval/_cli 2026-05-11T14:28:37,321 copying lm_eval/_cli/subcommand.py -> build/lib/lm_eval/_cli 2026-05-11T14:28:37,324 creating build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:28:37,325 copying lm_eval/tasks/nerel-bench/utils.py -> build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:28:37,330 creating build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T14:28:37,331 copying lm_eval/tasks/med_prescriptions/utils.py -> build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T14:28:37,338 creating build/lib/lm_eval/tasks/winogrande 2026-05-11T14:28:37,340 copying lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/lib/lm_eval/tasks/winogrande 2026-05-11T14:28:37,343 creating build/lib/lm_eval/tasks/humaneval 2026-05-11T14:28:37,344 copying lm_eval/tasks/humaneval/utils.py -> build/lib/lm_eval/tasks/humaneval 2026-05-11T14:28:37,347 creating build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:37,348 copying lm_eval/tasks/scrolls/task.py -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:37,351 creating build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:28:37,352 copying lm_eval/tasks/simple_cooccurrence_bias/utils.py -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:28:37,355 creating build/lib/lm_eval/tasks/glianorex 2026-05-11T14:28:37,356 copying lm_eval/tasks/glianorex/preprocess_glianorex.py -> build/lib/lm_eval/tasks/glianorex 2026-05-11T14:28:37,358 creating build/lib/lm_eval/tasks/discrim_eval 2026-05-11T14:28:37,359 copying lm_eval/tasks/discrim_eval/utils.py -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T14:28:37,361 creating build/lib/lm_eval/tasks/qa4mre 2026-05-11T14:28:37,362 copying lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T14:28:37,364 creating build/lib/lm_eval/tasks/race 2026-05-11T14:28:37,365 copying lm_eval/tasks/race/preprocess_race.py -> build/lib/lm_eval/tasks/race 2026-05-11T14:28:37,368 creating build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:37,369 copying lm_eval/tasks/winogender/utils.py -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:37,371 creating build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:37,372 copying lm_eval/tasks/arab_culture/prompts.py -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:37,375 copying lm_eval/tasks/arab_culture/utils_mcq.py -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:37,376 copying lm_eval/tasks/arab_culture/_generate_configs.py -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:37,379 creating build/lib/lm_eval/tasks/eus_trivia 2026-05-11T14:28:37,380 copying lm_eval/tasks/eus_trivia/utils.py -> build/lib/lm_eval/tasks/eus_trivia 2026-05-11T14:28:37,383 creating build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T14:28:37,384 copying lm_eval/tasks/jsonschema_bench/metrics.py -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T14:28:37,387 creating build/lib/lm_eval/tasks/eus_reading 2026-05-11T14:28:37,388 copying lm_eval/tasks/eus_reading/utils.py -> build/lib/lm_eval/tasks/eus_reading 2026-05-11T14:28:37,391 creating build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:37,391 copying lm_eval/tasks/aclue/_generate_configs.py -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:37,394 creating build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:37,395 copying lm_eval/tasks/careqa/utils_perplexity.py -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:37,397 copying lm_eval/tasks/careqa/utils_open.py -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:37,398 copying lm_eval/tasks/careqa/utils.py -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:37,401 creating build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T14:28:37,402 copying lm_eval/tasks/icelandic_winogrande/preprocess_winogrande.py -> build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T14:28:37,404 creating build/lib/lm_eval/tasks/copal_id 2026-05-11T14:28:37,405 copying lm_eval/tasks/copal_id/utils.py -> build/lib/lm_eval/tasks/copal_id 2026-05-11T14:28:37,408 creating build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:37,409 copying lm_eval/tasks/arab_culture_completion/utils_completion.py -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:37,411 copying lm_eval/tasks/arab_culture_completion/prompts.py -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:37,413 copying lm_eval/tasks/arab_culture_completion/_generate_configs.py -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:37,416 creating build/lib/lm_eval/tasks/medtext 2026-05-11T14:28:37,417 copying lm_eval/tasks/medtext/utils_perplexity.py -> build/lib/lm_eval/tasks/medtext 2026-05-11T14:28:37,419 copying lm_eval/tasks/medtext/utils.py -> build/lib/lm_eval/tasks/medtext 2026-05-11T14:28:37,421 creating build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:37,422 copying lm_eval/tasks/pisa/utils.py -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:37,425 creating build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:37,426 copying lm_eval/tasks/xnli/utils.py -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:37,429 creating build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T14:28:37,431 copying lm_eval/tasks/openai-mmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T14:28:37,434 creating build/lib/lm_eval/tasks/meqsum 2026-05-11T14:28:37,435 copying lm_eval/tasks/meqsum/utils.py -> build/lib/lm_eval/tasks/meqsum 2026-05-11T14:28:37,440 creating build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:37,441 copying lm_eval/tasks/mlqa/generate_tasks.py -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:37,443 copying lm_eval/tasks/mlqa/utils.py -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:37,446 creating build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:37,446 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:37,448 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:37,450 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:37,452 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:37,455 creating build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,456 copying lm_eval/tasks/ruler/common_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,458 copying lm_eval/tasks/ruler/fwe_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,460 copying lm_eval/tasks/ruler/niah_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,463 copying lm_eval/tasks/ruler/qa_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,465 copying lm_eval/tasks/ruler/prepare_niah.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,467 copying lm_eval/tasks/ruler/vt_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,470 copying lm_eval/tasks/ruler/essays.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,472 copying lm_eval/tasks/ruler/cwe_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:37,474 creating build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T14:28:37,475 copying lm_eval/tasks/mmlu_prox/mmlu_prox_lite_config_generator.py -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T14:28:37,478 copying lm_eval/tasks/mmlu_prox/lang_libs.py -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T14:28:37,481 copying lm_eval/tasks/mmlu_prox/mmlu_prox_config_generator.py -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T14:28:37,483 creating build/lib/lm_eval/tasks/swde 2026-05-11T14:28:37,484 copying lm_eval/tasks/swde/task.py -> build/lib/lm_eval/tasks/swde 2026-05-11T14:28:37,487 creating build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:28:37,488 copying lm_eval/tasks/arabicmmlu/utils.py -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:28:37,490 copying lm_eval/tasks/arabicmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:28:37,493 creating build/lib/lm_eval/tasks/noticia 2026-05-11T14:28:37,494 copying lm_eval/tasks/noticia/utils.py -> build/lib/lm_eval/tasks/noticia 2026-05-11T14:28:37,497 creating build/lib/lm_eval/tasks/bigbench 2026-05-11T14:28:37,498 copying lm_eval/tasks/bigbench/generate_tasks.py -> build/lib/lm_eval/tasks/bigbench 2026-05-11T14:28:37,500 copying lm_eval/tasks/bigbench/utils.py -> build/lib/lm_eval/tasks/bigbench 2026-05-11T14:28:37,502 copying lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/lib/lm_eval/tasks/bigbench 2026-05-11T14:28:37,504 creating build/lib/lm_eval/tasks/bbh 2026-05-11T14:28:37,506 copying lm_eval/tasks/bbh/_generate_configs.py -> build/lib/lm_eval/tasks/bbh 2026-05-11T14:28:37,508 creating build/lib/lm_eval/tasks/drop 2026-05-11T14:28:37,509 copying lm_eval/tasks/drop/utils.py -> build/lib/lm_eval/tasks/drop 2026-05-11T14:28:37,512 creating build/lib/lm_eval/tasks/toxigen 2026-05-11T14:28:37,513 copying lm_eval/tasks/toxigen/utils.py -> build/lib/lm_eval/tasks/toxigen 2026-05-11T14:28:37,515 creating build/lib/lm_eval/tasks/pubmedqa 2026-05-11T14:28:37,516 copying lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/lib/lm_eval/tasks/pubmedqa 2026-05-11T14:28:37,518 creating build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:28:37,519 copying lm_eval/tasks/infinitebench/utils.py -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:28:37,522 creating build/lib/lm_eval/tasks/agieval 2026-05-11T14:28:37,523 copying lm_eval/tasks/agieval/utils.py -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:28:37,526 creating build/lib/lm_eval/tasks/unitxt 2026-05-11T14:28:37,527 copying lm_eval/tasks/unitxt/task.py -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:28:37,530 creating build/lib/lm_eval/tasks/metabench 2026-05-11T14:28:37,531 copying lm_eval/tasks/metabench/process_docs_permute.py -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:28:37,534 copying lm_eval/tasks/metabench/process_docs.py -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:28:37,537 creating build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:28:37,538 copying lm_eval/tasks/cmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:28:37,541 creating build/lib/lm_eval/tasks/medmcqa 2026-05-11T14:28:37,542 copying lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/lib/lm_eval/tasks/medmcqa 2026-05-11T14:28:37,546 creating build/lib/lm_eval/tasks/paws-x 2026-05-11T14:28:37,547 copying lm_eval/tasks/paws-x/_generate_config.py -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:28:37,550 copying lm_eval/tasks/paws-x/utils.py -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:28:37,552 creating build/lib/lm_eval/tasks/blimp 2026-05-11T14:28:37,553 copying lm_eval/tasks/blimp/generate_configs.py -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:28:37,556 creating build/lib/lm_eval/tasks/mmmu 2026-05-11T14:28:37,557 copying lm_eval/tasks/mmmu/utils.py -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:28:37,561 creating build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:28:37,562 copying lm_eval/tasks/egymmlu/utils.py -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:28:37,564 copying lm_eval/tasks/egymmlu/_generate_configs.py -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:28:37,566 creating build/lib/lm_eval/tasks/moral_stories 2026-05-11T14:28:37,567 copying lm_eval/tasks/moral_stories/utils.py -> build/lib/lm_eval/tasks/moral_stories 2026-05-11T14:28:37,570 creating build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:28:37,571 copying lm_eval/tasks/med_concepts_qa/_generate_configs.py -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:28:37,573 creating build/lib/lm_eval/tasks/babilong 2026-05-11T14:28:37,574 copying lm_eval/tasks/babilong/common_utils.py -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:28:37,578 creating build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:28:37,578 copying lm_eval/tasks/tinyBenchmarks/utils_winogrande.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:28:37,580 copying lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:28:37,582 copying lm_eval/tasks/tinyBenchmarks/agg_functions.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:28:37,584 copying lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:28:37,587 creating build/lib/lm_eval/tasks/chartqa 2026-05-11T14:28:37,588 copying lm_eval/tasks/chartqa/utils.py -> build/lib/lm_eval/tasks/chartqa 2026-05-11T14:28:37,591 creating build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:28:37,592 copying lm_eval/tasks/minerva_math/utils.py -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:28:37,597 creating build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:28:37,598 copying lm_eval/tasks/hendrycks_ethics/utils.py -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:28:37,600 creating build/lib/lm_eval/tasks/squad_completion 2026-05-11T14:28:37,601 copying lm_eval/tasks/squad_completion/task.py -> build/lib/lm_eval/tasks/squad_completion 2026-05-11T14:28:37,604 creating build/lib/lm_eval/tasks/belebele 2026-05-11T14:28:37,605 copying lm_eval/tasks/belebele/_generate_configs.py -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:28:37,607 creating build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:28:37,608 copying lm_eval/tasks/basque_bench/utils.py -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:28:37,610 creating build/lib/lm_eval/tasks/kobest 2026-05-11T14:28:37,611 copying lm_eval/tasks/kobest/utils.py -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:28:37,614 creating build/lib/lm_eval/tasks/logiqa 2026-05-11T14:28:37,615 copying lm_eval/tasks/logiqa/utils_logiqa.py -> build/lib/lm_eval/tasks/logiqa 2026-05-11T14:28:37,619 creating build/lib/lm_eval/tasks/mutual 2026-05-11T14:28:37,621 copying lm_eval/tasks/mutual/utils.py -> build/lib/lm_eval/tasks/mutual 2026-05-11T14:28:37,624 creating build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:28:37,625 copying lm_eval/tasks/galician_bench/utils.py -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:28:37,628 creating build/lib/lm_eval/tasks/xcopa 2026-05-11T14:28:37,629 copying lm_eval/tasks/xcopa/utils.py -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:28:37,632 creating build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T14:28:37,633 copying lm_eval/tasks/cnn_dailymail/utils.py -> build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T14:28:37,636 creating build/lib/lm_eval/tasks/ifeval 2026-05-11T14:28:37,637 copying lm_eval/tasks/ifeval/instructions_registry.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T14:28:37,640 copying lm_eval/tasks/ifeval/instructions_util.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T14:28:37,643 copying lm_eval/tasks/ifeval/instructions.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T14:28:37,646 copying lm_eval/tasks/ifeval/utils.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T14:28:37,650 creating build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T14:28:37,651 copying lm_eval/tasks/mediqa_qa2019/utils_perplexity.py -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T14:28:37,653 copying lm_eval/tasks/mediqa_qa2019/utils.py -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T14:28:37,655 creating build/lib/lm_eval/tasks/csatqa 2026-05-11T14:28:37,656 copying lm_eval/tasks/csatqa/utils.py -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:28:37,658 copying lm_eval/tasks/csatqa/_generate_configs.py -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:28:37,661 creating build/lib/lm_eval/tasks/graphwalks 2026-05-11T14:28:37,662 copying lm_eval/tasks/graphwalks/utils.py -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T14:28:37,666 creating build/lib/lm_eval/tasks/afrixnli 2026-05-11T14:28:37,667 copying lm_eval/tasks/afrixnli/utils.py -> build/lib/lm_eval/tasks/afrixnli 2026-05-11T14:28:37,670 copying lm_eval/tasks/afrixnli/gen_utils.py -> build/lib/lm_eval/tasks/afrixnli 2026-05-11T14:28:37,674 creating build/lib/lm_eval/tasks/c4 2026-05-11T14:28:37,675 copying lm_eval/tasks/c4/preprocess_c4.py -> build/lib/lm_eval/tasks/c4 2026-05-11T14:28:37,677 creating build/lib/lm_eval/tasks/toksuite 2026-05-11T14:28:37,678 copying lm_eval/tasks/toksuite/utils.py -> build/lib/lm_eval/tasks/toksuite 2026-05-11T14:28:37,682 creating build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:28:37,683 copying lm_eval/tasks/jfinqa/test_jfinqa_utils.py -> build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:28:37,685 copying lm_eval/tasks/jfinqa/utils.py -> build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:28:37,689 creating build/lib/lm_eval/tasks/wmt2016 2026-05-11T14:28:37,690 copying lm_eval/tasks/wmt2016/metrics.py -> build/lib/lm_eval/tasks/wmt2016 2026-05-11T14:28:37,695 creating build/lib/lm_eval/tasks/ceval 2026-05-11T14:28:37,696 copying lm_eval/tasks/ceval/_generate_configs.py -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:28:37,699 creating build/lib/lm_eval/tasks/coqa 2026-05-11T14:28:37,700 copying lm_eval/tasks/coqa/utils.py -> build/lib/lm_eval/tasks/coqa 2026-05-11T14:28:37,703 creating build/lib/lm_eval/tasks/olaph 2026-05-11T14:28:37,704 copying lm_eval/tasks/olaph/utils_perplexity.py -> build/lib/lm_eval/tasks/olaph 2026-05-11T14:28:37,706 copying lm_eval/tasks/olaph/utils.py -> build/lib/lm_eval/tasks/olaph 2026-05-11T14:28:37,708 creating build/lib/lm_eval/tasks/afrimmlu 2026-05-11T14:28:37,709 copying lm_eval/tasks/afrimmlu/utils.py -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T14:28:37,711 copying lm_eval/tasks/afrimmlu/gen_utils.py -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T14:28:37,714 creating build/lib/lm_eval/tasks/mbpp 2026-05-11T14:28:37,714 copying lm_eval/tasks/mbpp/utils.py -> build/lib/lm_eval/tasks/mbpp 2026-05-11T14:28:37,718 creating build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:28:37,718 copying lm_eval/tasks/catalan_bench/utils.py -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:28:37,721 creating build/lib/lm_eval/tasks/bbq 2026-05-11T14:28:37,722 copying lm_eval/tasks/bbq/utils.py -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:28:37,725 creating build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:28:37,726 copying lm_eval/tasks/humaneval_infilling/utils.py -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:28:37,729 creating build/lib/lm_eval/tasks/mgsm 2026-05-11T14:28:37,730 copying lm_eval/tasks/mgsm/utils.py -> build/lib/lm_eval/tasks/mgsm 2026-05-11T14:28:37,733 creating build/lib/lm_eval/tasks/medqa 2026-05-11T14:28:37,734 copying lm_eval/tasks/medqa/preprocess_medqa.py -> build/lib/lm_eval/tasks/medqa 2026-05-11T14:28:37,736 creating build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T14:28:37,737 copying lm_eval/tasks/mimic_repsum/utils_perplexity.py -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T14:28:37,739 copying lm_eval/tasks/mimic_repsum/utils.py -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T14:28:37,742 creating build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:28:37,743 copying lm_eval/tasks/xwinograd/utils.py -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:28:37,745 creating build/lib/lm_eval/tasks/qasper 2026-05-11T14:28:37,746 copying lm_eval/tasks/qasper/metrics.py -> build/lib/lm_eval/tasks/qasper 2026-05-11T14:28:37,748 copying lm_eval/tasks/qasper/utils.py -> build/lib/lm_eval/tasks/qasper 2026-05-11T14:28:37,751 creating build/lib/lm_eval/tasks/afrimgsm 2026-05-11T14:28:37,752 copying lm_eval/tasks/afrimgsm/utils.py -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T14:28:37,754 copying lm_eval/tasks/afrimgsm/gen_utils.py -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T14:28:37,757 creating build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:28:37,758 copying lm_eval/tasks/crows_pairs/utils.py -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:28:37,761 creating build/lib/lm_eval/tasks/libra 2026-05-11T14:28:37,762 copying lm_eval/tasks/libra/utils.py -> build/lib/lm_eval/tasks/libra 2026-05-11T14:28:37,766 creating build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:28:37,767 copying lm_eval/tasks/slr_bench/lm_eval_slr_bench.py -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:28:37,769 creating build/lib/lm_eval/tasks/logiqa2 2026-05-11T14:28:37,770 copying lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T14:28:37,773 creating build/lib/lm_eval/tasks/longbench 2026-05-11T14:28:37,774 copying lm_eval/tasks/longbench/_generate_config.py -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:28:37,776 copying lm_eval/tasks/longbench/metrics.py -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:28:37,779 copying lm_eval/tasks/longbench/utils.py -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:28:37,782 creating build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:28:37,783 copying lm_eval/tasks/basqueglue/utils.py -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:28:37,785 creating build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:28:37,787 copying lm_eval/tasks/cruxeval/utils.py -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:28:37,789 creating build/lib/lm_eval/tasks/mathqa 2026-05-11T14:28:37,790 copying lm_eval/tasks/mathqa/utils.py -> build/lib/lm_eval/tasks/mathqa 2026-05-11T14:28:37,794 creating build/lib/lm_eval/tasks/hellaswag 2026-05-11T14:28:37,795 copying lm_eval/tasks/hellaswag/utils.py -> build/lib/lm_eval/tasks/hellaswag 2026-05-11T14:28:37,799 creating build/lib/lm_eval/tasks/xquad 2026-05-11T14:28:37,800 copying lm_eval/tasks/xquad/utils.py -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:28:37,802 creating build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:28:37,803 copying lm_eval/tasks/spanish_bench/utils.py -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:28:37,805 creating build/lib/lm_eval/tasks/paloma 2026-05-11T14:28:37,806 copying lm_eval/tasks/paloma/paloma_utils.py -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:28:37,808 creating build/lib/lm_eval/tasks/med_text_classification 2026-05-11T14:28:37,809 copying lm_eval/tasks/med_text_classification/utils.py -> build/lib/lm_eval/tasks/med_text_classification 2026-05-11T14:28:37,812 creating build/lib/lm_eval/tasks/translation 2026-05-11T14:28:37,813 copying lm_eval/tasks/translation/utils.py -> build/lib/lm_eval/tasks/translation 2026-05-11T14:28:37,815 creating build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:28:37,816 copying lm_eval/tasks/truthfulqa-multi/utils.py -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:28:37,819 creating build/lib/lm_eval/tasks/cabbq 2026-05-11T14:28:37,820 copying lm_eval/tasks/cabbq/utils.py -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:28:37,823 creating build/lib/lm_eval/tasks/common_voice 2026-05-11T14:28:37,824 copying lm_eval/tasks/common_voice/utils.py -> build/lib/lm_eval/tasks/common_voice 2026-05-11T14:28:37,826 creating build/lib/lm_eval/tasks/groundcocoa 2026-05-11T14:28:37,827 copying lm_eval/tasks/groundcocoa/utils.py -> build/lib/lm_eval/tasks/groundcocoa 2026-05-11T14:28:37,830 creating build/lib/lm_eval/tasks/fda 2026-05-11T14:28:37,831 copying lm_eval/tasks/fda/task.py -> build/lib/lm_eval/tasks/fda 2026-05-11T14:28:37,833 creating build/lib/lm_eval/tasks/squadv2 2026-05-11T14:28:37,834 copying lm_eval/tasks/squadv2/task.py -> build/lib/lm_eval/tasks/squadv2 2026-05-11T14:28:37,837 creating build/lib/lm_eval/tasks/aime 2026-05-11T14:28:37,838 copying lm_eval/tasks/aime/utils.py -> build/lib/lm_eval/tasks/aime 2026-05-11T14:28:37,841 creating build/lib/lm_eval/tasks/french_bench 2026-05-11T14:28:37,842 copying lm_eval/tasks/french_bench/preprocess_wikitext.py -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:28:37,844 copying lm_eval/tasks/french_bench/utils.py -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:28:37,846 creating build/lib/lm_eval/tasks/eq_bench 2026-05-11T14:28:37,847 copying lm_eval/tasks/eq_bench/utils.py -> build/lib/lm_eval/tasks/eq_bench 2026-05-11T14:28:37,850 creating build/lib/lm_eval/tasks/esbbq 2026-05-11T14:28:37,851 copying lm_eval/tasks/esbbq/utils.py -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:28:37,854 creating build/lib/lm_eval/tasks/realtoxicityprompts 2026-05-11T14:28:37,855 copying lm_eval/tasks/realtoxicityprompts/metric.py -> build/lib/lm_eval/tasks/realtoxicityprompts 2026-05-11T14:28:37,858 creating build/lib/lm_eval/tasks/webqs 2026-05-11T14:28:37,859 copying lm_eval/tasks/webqs/utils.py -> build/lib/lm_eval/tasks/webqs 2026-05-11T14:28:37,863 creating build/lib/lm_eval/tasks/mts_dialog 2026-05-11T14:28:37,864 copying lm_eval/tasks/mts_dialog/utils_perplexity.py -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T14:28:37,866 copying lm_eval/tasks/mts_dialog/utils.py -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T14:28:37,868 creating build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:28:37,869 copying lm_eval/tasks/darijammlu/utils.py -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:28:37,871 copying lm_eval/tasks/darijammlu/_generate_configs.py -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:28:37,873 creating build/lib/lm_eval/tasks/truthfulqa 2026-05-11T14:28:37,874 copying lm_eval/tasks/truthfulqa/utils.py -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T14:28:37,877 creating build/lib/lm_eval/tasks/meddialog 2026-05-11T14:28:37,878 copying lm_eval/tasks/meddialog/utils_perplexity.py -> build/lib/lm_eval/tasks/meddialog 2026-05-11T14:28:37,880 copying lm_eval/tasks/meddialog/utils.py -> build/lib/lm_eval/tasks/meddialog 2026-05-11T14:28:37,882 creating build/lib/lm_eval/tasks/histoires_morales 2026-05-11T14:28:37,883 copying lm_eval/tasks/histoires_morales/utils.py -> build/lib/lm_eval/tasks/histoires_morales 2026-05-11T14:28:37,885 creating build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T14:28:37,886 copying lm_eval/tasks/darijahellaswag/utils.py -> build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T14:28:37,889 creating build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:28:37,890 copying lm_eval/tasks/mmlu-pro-plus/utils.py -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:28:37,892 creating build/lib/lm_eval/tasks/wsc273 2026-05-11T14:28:37,893 copying lm_eval/tasks/wsc273/utils.py -> build/lib/lm_eval/tasks/wsc273 2026-05-11T14:28:37,896 creating build/lib/lm_eval/tasks/score 2026-05-11T14:28:37,897 copying lm_eval/tasks/score/non_greedy_summarizer.py -> build/lib/lm_eval/tasks/score 2026-05-11T14:28:37,900 copying lm_eval/tasks/score/utils.py -> build/lib/lm_eval/tasks/score 2026-05-11T14:28:37,903 creating build/lib/lm_eval/tasks/mmlu 2026-05-11T14:28:37,904 copying lm_eval/tasks/mmlu/_generate_configs.py -> build/lib/lm_eval/tasks/mmlu 2026-05-11T14:28:37,907 creating build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:28:37,908 copying lm_eval/tasks/evalita_llm/metrics.py -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:28:37,911 copying lm_eval/tasks/evalita_llm/utils.py -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:28:37,915 copying lm_eval/tasks/evalita_llm/sum_utils.py -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:28:37,919 creating build/lib/lm_eval/tasks/lingoly 2026-05-11T14:28:37,920 copying lm_eval/tasks/lingoly/utils.py -> build/lib/lm_eval/tasks/lingoly 2026-05-11T14:28:37,922 copying lm_eval/tasks/lingoly/script.py -> build/lib/lm_eval/tasks/lingoly 2026-05-11T14:28:37,924 creating build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:28:37,925 copying lm_eval/tasks/hendrycks_math/utils.py -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:28:37,928 creating build/lib/lm_eval/tasks/mmlusr 2026-05-11T14:28:37,929 copying lm_eval/tasks/mmlusr/config.py -> build/lib/lm_eval/tasks/mmlusr 2026-05-11T14:28:37,932 creating build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T14:28:37,933 copying lm_eval/tasks/egyhellaswag/utils.py -> build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T14:28:37,935 creating build/lib/lm_eval/tasks/wikitext 2026-05-11T14:28:37,936 copying lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/lib/lm_eval/tasks/wikitext 2026-05-11T14:28:37,939 creating build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:28:37,940 copying lm_eval/tasks/mmlu_pro/utils.py -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:28:37,942 creating build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:28:37,943 copying lm_eval/tasks/eus_exams/configs.py -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:28:37,946 copying lm_eval/tasks/eus_exams/utils.py -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:28:37,948 creating build/lib/lm_eval/tasks/glue/mnli 2026-05-11T14:28:37,950 copying lm_eval/tasks/glue/mnli/utils.py -> build/lib/lm_eval/tasks/glue/mnli 2026-05-11T14:28:37,957 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:28:37,959 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:28:37,961 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:28:37,962 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:28:37,965 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:28:37,966 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:28:37,968 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:28:37,969 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:28:37,971 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:28:37,972 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:28:37,974 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:28:37,976 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:28:37,978 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:28:37,979 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:28:37,981 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:28:37,982 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:28:37,985 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:37,986 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:37,988 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:37,989 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:37,991 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:28:37,992 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:28:37,995 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:28:37,996 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:28:37,998 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:38,000 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:38,002 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:28:38,003 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:28:38,005 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:28:38,006 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:28:38,009 creating build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:38,010 copying lm_eval/tasks/acpbench/gen_2shot/acp_utils.py -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:38,014 creating build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:38,015 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_utils.py -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:38,019 creating build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:38,020 copying lm_eval/tasks/hrm8k/en/utils.py -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:38,023 creating build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:38,024 copying lm_eval/tasks/hrm8k/default/utils.py -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:38,030 creating build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:38,031 copying lm_eval/tasks/openai-mmmlu/default/utils.py -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:38,034 creating build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:38,036 copying lm_eval/tasks/global_mmlu/full/de/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:38,039 creating build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:38,040 copying lm_eval/tasks/global_mmlu/full/hi/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:38,043 creating build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:38,044 copying lm_eval/tasks/global_mmlu/full/bn/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:38,046 creating build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:38,048 copying lm_eval/tasks/global_mmlu/full/ar/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:38,050 creating build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:38,051 copying lm_eval/tasks/global_mmlu/full/es/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:38,054 creating build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:38,055 copying lm_eval/tasks/global_mmlu/full/pt/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:38,058 creating build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:38,059 copying lm_eval/tasks/global_mmlu/full/fil/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:38,061 creating build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:38,062 copying lm_eval/tasks/global_mmlu/full/lt/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:38,065 creating build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:38,066 copying lm_eval/tasks/global_mmlu/full/cs/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:38,069 creating build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:38,070 copying lm_eval/tasks/global_mmlu/full/ro/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:38,073 creating build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:38,074 copying lm_eval/tasks/global_mmlu/full/ru/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:38,077 creating build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:38,078 copying lm_eval/tasks/global_mmlu/full/sr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:38,081 creating build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:38,082 copying lm_eval/tasks/global_mmlu/full/el/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:38,085 creating build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:38,086 copying lm_eval/tasks/global_mmlu/full/zh/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:38,088 creating build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:38,089 copying lm_eval/tasks/global_mmlu/full/mg/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:38,092 creating build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:38,093 copying lm_eval/tasks/global_mmlu/full/id/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:38,096 creating build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:38,097 copying lm_eval/tasks/global_mmlu/full/ha/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:38,100 creating build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:38,101 copying lm_eval/tasks/global_mmlu/full/ko/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:38,103 creating build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:38,104 copying lm_eval/tasks/global_mmlu/full/ja/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:38,107 creating build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:38,108 copying lm_eval/tasks/global_mmlu/full/pl/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:38,111 creating build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:38,112 copying lm_eval/tasks/global_mmlu/full/uk/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:38,114 creating build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:38,115 copying lm_eval/tasks/global_mmlu/full/ig/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:38,118 creating build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:38,119 copying lm_eval/tasks/global_mmlu/full/sw/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:38,122 creating build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:38,123 copying lm_eval/tasks/global_mmlu/full/sn/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:38,125 creating build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:38,126 copying lm_eval/tasks/global_mmlu/full/fr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:38,129 creating build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:38,130 copying lm_eval/tasks/global_mmlu/full/en/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:38,133 creating build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:38,134 copying lm_eval/tasks/global_mmlu/full/ne/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:38,136 creating build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:38,137 copying lm_eval/tasks/global_mmlu/full/am/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:38,140 creating build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:38,141 copying lm_eval/tasks/global_mmlu/full/yo/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:38,144 creating build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:38,145 copying lm_eval/tasks/global_mmlu/full/te/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:38,148 creating build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:38,149 copying lm_eval/tasks/global_mmlu/full/ms/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:38,152 creating build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:38,153 copying lm_eval/tasks/global_mmlu/full/he/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:38,155 creating build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:38,157 copying lm_eval/tasks/global_mmlu/full/sv/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:38,159 creating build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:38,160 copying lm_eval/tasks/global_mmlu/full/vi/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:38,163 creating build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:38,163 copying lm_eval/tasks/global_mmlu/full/ky/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:38,166 creating build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:38,168 copying lm_eval/tasks/global_mmlu/full/fa/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:38,170 creating build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:38,171 copying lm_eval/tasks/global_mmlu/full/nl/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:38,174 creating build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:38,174 copying lm_eval/tasks/global_mmlu/full/ny/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:38,177 creating build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:38,178 copying lm_eval/tasks/global_mmlu/full/tr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:38,181 creating build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:38,182 copying lm_eval/tasks/global_mmlu/full/it/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:38,184 creating build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:38,185 copying lm_eval/tasks/global_mmlu/full/so/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:38,188 creating build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:38,189 copying lm_eval/tasks/global_mmlu/full/si/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:38,191 creating build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:38,193 copying lm_eval/tasks/global_mmlu/default/de/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:38,195 creating build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:38,196 copying lm_eval/tasks/global_mmlu/default/hi/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:38,198 creating build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:38,199 copying lm_eval/tasks/global_mmlu/default/bn/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:38,202 creating build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:38,203 copying lm_eval/tasks/global_mmlu/default/ar/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:38,205 creating build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:38,206 copying lm_eval/tasks/global_mmlu/default/es/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:38,209 creating build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:38,210 copying lm_eval/tasks/global_mmlu/default/pt/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:38,213 creating build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:38,214 copying lm_eval/tasks/global_mmlu/default/zh/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:38,217 creating build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:38,218 copying lm_eval/tasks/global_mmlu/default/id/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:38,220 creating build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:38,222 copying lm_eval/tasks/global_mmlu/default/ko/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:38,225 creating build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:38,226 copying lm_eval/tasks/global_mmlu/default/ja/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:38,229 creating build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:38,230 copying lm_eval/tasks/global_mmlu/default/sw/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:38,233 creating build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:38,234 copying lm_eval/tasks/global_mmlu/default/fr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:38,236 creating build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:38,237 copying lm_eval/tasks/global_mmlu/default/en/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:38,239 creating build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:38,240 copying lm_eval/tasks/global_mmlu/default/yo/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:38,243 creating build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:38,244 copying lm_eval/tasks/global_mmlu/default/it/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:38,246 creating build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T14:28:38,248 copying lm_eval/tasks/gpqa/generative/utils.py -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T14:28:38,250 copying lm_eval/tasks/gpqa/generative/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T14:28:38,252 creating build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:28:38,253 copying lm_eval/tasks/gpqa/cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:28:38,255 copying lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:28:38,257 creating build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:28:38,258 copying lm_eval/tasks/gpqa/n_shot/utils.py -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:28:38,260 copying lm_eval/tasks/gpqa/n_shot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:28:38,263 creating build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:28:38,264 copying lm_eval/tasks/gpqa/cot_n_shot/utils.py -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:28:38,266 copying lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:28:38,268 creating build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:28:38,269 copying lm_eval/tasks/gpqa/zeroshot/utils.py -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:28:38,271 copying lm_eval/tasks/gpqa/zeroshot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:28:38,273 creating build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:38,274 copying lm_eval/tasks/portuguese_bench/flores_pt/create_yamls_flores_pt.py -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:38,277 creating build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:38,278 copying lm_eval/tasks/mmlu_prox/de/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:38,281 creating build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:38,282 copying lm_eval/tasks/mmlu_prox/hi/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:38,284 creating build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:38,285 copying lm_eval/tasks/mmlu_prox/ur/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:38,288 creating build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:38,289 copying lm_eval/tasks/mmlu_prox/bn/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:38,291 creating build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:38,292 copying lm_eval/tasks/mmlu_prox/ar/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:38,295 creating build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:38,296 copying lm_eval/tasks/mmlu_prox/es/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:38,298 creating build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:38,300 copying lm_eval/tasks/mmlu_prox/pt/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:38,302 creating build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:38,303 copying lm_eval/tasks/mmlu_prox/zu/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:38,305 creating build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:38,306 copying lm_eval/tasks/mmlu_prox/cs/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:38,309 creating build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:38,310 copying lm_eval/tasks/mmlu_prox/mr/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:38,312 creating build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:38,313 copying lm_eval/tasks/mmlu_prox/af/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:38,316 creating build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:38,317 copying lm_eval/tasks/mmlu_prox/wo/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:38,319 creating build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:38,320 copying lm_eval/tasks/mmlu_prox/th/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:38,322 creating build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:38,323 copying lm_eval/tasks/mmlu_prox/ru/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:38,326 creating build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:38,327 copying lm_eval/tasks/mmlu_prox/sr/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:38,329 creating build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:38,330 copying lm_eval/tasks/mmlu_prox/zh/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:38,332 creating build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:38,333 copying lm_eval/tasks/mmlu_prox/id/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:38,336 creating build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:38,337 copying lm_eval/tasks/mmlu_prox/ko/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:38,339 creating build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:38,340 copying lm_eval/tasks/mmlu_prox/ja/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:38,343 creating build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:38,344 copying lm_eval/tasks/mmlu_prox/uk/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:38,346 creating build/lib/lm_eval/tasks/mmlu_prox/template 2026-05-11T14:28:38,347 copying lm_eval/tasks/mmlu_prox/template/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/template 2026-05-11T14:28:38,349 creating build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:38,350 copying lm_eval/tasks/mmlu_prox/sw/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:38,353 creating build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:38,354 copying lm_eval/tasks/mmlu_prox/fr/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:38,356 creating build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:38,357 copying lm_eval/tasks/mmlu_prox/en/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:38,360 creating build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:38,361 copying lm_eval/tasks/mmlu_prox/ne/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:38,363 creating build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:38,364 copying lm_eval/tasks/mmlu_prox/yo/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:38,367 creating build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:38,368 copying lm_eval/tasks/mmlu_prox/te/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:38,370 creating build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:38,371 copying lm_eval/tasks/mmlu_prox/vi/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:38,373 creating build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:38,374 copying lm_eval/tasks/mmlu_prox/hu/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:38,377 creating build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:38,378 copying lm_eval/tasks/mmlu_prox/it/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:38,384 creating build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:28:38,386 copying lm_eval/tasks/global_piqa/completions/_generate_config.py -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:28:38,388 creating build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:28:38,389 copying lm_eval/tasks/global_piqa/prompted/_generate_config.py -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:28:38,393 creating build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:28:38,394 copying lm_eval/tasks/bbh/cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:28:38,396 creating build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:28:38,397 copying lm_eval/tasks/bbh/zeroshot/utils.py -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:28:38,401 creating build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:28:38,402 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:28:38,405 creating build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:28:38,406 copying lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:28:38,409 creating build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:28:38,411 copying lm_eval/tasks/include/few_shot_en/Indonesian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:28:38,413 creating build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:28:38,414 copying lm_eval/tasks/include/few_shot_en/Tagalog/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:28:38,417 creating build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:28:38,418 copying lm_eval/tasks/include/few_shot_en/North Macedonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:28:38,420 creating build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:28:38,421 copying lm_eval/tasks/include/few_shot_en/Malayalam/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:28:38,423 creating build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:28:38,424 copying lm_eval/tasks/include/few_shot_en/Belarusian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:28:38,426 creating build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:28:38,427 copying lm_eval/tasks/include/few_shot_en/Kazakh/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:28:38,430 creating build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:28:38,431 copying lm_eval/tasks/include/few_shot_en/Persian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:28:38,433 creating build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:28:38,434 copying lm_eval/tasks/include/few_shot_en/Arabic/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:28:38,437 creating build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:28:38,437 copying lm_eval/tasks/include/few_shot_en/Ukrainian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:28:38,440 creating build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:28:38,441 copying lm_eval/tasks/include/few_shot_en/Telugu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:28:38,443 creating build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:28:38,444 copying lm_eval/tasks/include/few_shot_en/Urdu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:28:38,446 creating build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:28:38,447 copying lm_eval/tasks/include/few_shot_en/Lithuanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:28:38,449 creating build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:28:38,450 copying lm_eval/tasks/include/few_shot_en/Vietnamese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:28:38,453 creating build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:28:38,454 copying lm_eval/tasks/include/few_shot_en/Albanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:28:38,456 creating build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:28:38,457 copying lm_eval/tasks/include/few_shot_en/Finnish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:28:38,460 creating build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:28:38,460 copying lm_eval/tasks/include/few_shot_en/Spanish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:28:38,463 creating build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:28:38,464 copying lm_eval/tasks/include/few_shot_en/Nepali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:28:38,466 creating build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:28:38,467 copying lm_eval/tasks/include/few_shot_en/Polish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:28:38,470 creating build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:28:38,471 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:28:38,473 creating build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:28:38,474 copying lm_eval/tasks/include/few_shot_en/Serbian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:28:38,476 creating build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:28:38,477 copying lm_eval/tasks/include/few_shot_en/Portuguese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:28:38,480 creating build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:28:38,481 copying lm_eval/tasks/include/few_shot_en/French/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:28:38,483 creating build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:28:38,484 copying lm_eval/tasks/include/few_shot_en/Uzbek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:28:38,486 creating build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:28:38,487 copying lm_eval/tasks/include/few_shot_en/Malay/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:28:38,490 creating build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:28:38,491 copying lm_eval/tasks/include/few_shot_en/Italian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:28:38,493 creating build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:28:38,494 copying lm_eval/tasks/include/few_shot_en/Estonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:28:38,497 creating build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:28:38,498 copying lm_eval/tasks/include/few_shot_en/Bengali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:28:38,500 creating build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:28:38,501 copying lm_eval/tasks/include/few_shot_en/Georgian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:28:38,503 creating build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:28:38,504 copying lm_eval/tasks/include/few_shot_en/Russian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:28:38,507 creating build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:28:38,508 copying lm_eval/tasks/include/few_shot_en/Korean/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:28:38,510 creating build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:28:38,511 copying lm_eval/tasks/include/few_shot_en/Chinese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:28:38,514 creating build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:28:38,514 copying lm_eval/tasks/include/few_shot_en/Croatian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:28:38,517 creating build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:28:38,518 copying lm_eval/tasks/include/few_shot_en/Dutch/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:28:38,520 creating build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:28:38,521 copying lm_eval/tasks/include/few_shot_en/Hindi/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:28:38,523 creating build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:28:38,524 copying lm_eval/tasks/include/few_shot_en/Tamil/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:28:38,526 creating build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:28:38,527 copying lm_eval/tasks/include/few_shot_en/Hungarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:28:38,530 creating build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:28:38,531 copying lm_eval/tasks/include/few_shot_en/Turkish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:28:38,533 creating build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:28:38,534 copying lm_eval/tasks/include/few_shot_en/Greek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:28:38,536 creating build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:28:38,537 copying lm_eval/tasks/include/few_shot_en/Armenian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:28:38,540 creating build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:28:38,541 copying lm_eval/tasks/include/few_shot_en/Hebrew/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:28:38,543 creating build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:28:38,545 copying lm_eval/tasks/include/few_shot_en/German/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:28:38,547 creating build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:28:38,548 copying lm_eval/tasks/include/few_shot_en/Japanese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:28:38,551 creating build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:28:38,552 copying lm_eval/tasks/include/few_shot_en/Basque/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:28:38,554 creating build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:28:38,555 copying lm_eval/tasks/include/few_shot_en/Bulgarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:28:38,558 creating build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:28:38,559 copying lm_eval/tasks/include/default/Indonesian/utils.py -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:28:38,562 creating build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T14:28:38,563 copying lm_eval/tasks/include/default/Tagalog/utils.py -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T14:28:38,565 creating build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:28:38,566 copying lm_eval/tasks/include/default/North Macedonian/utils.py -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:28:38,568 creating build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:28:38,569 copying lm_eval/tasks/include/default/Malayalam/utils.py -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:28:38,572 creating build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:28:38,573 copying lm_eval/tasks/include/default/Belarusian/utils.py -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:28:38,575 creating build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T14:28:38,576 copying lm_eval/tasks/include/default/Kazakh/utils.py -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T14:28:38,578 creating build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:28:38,579 copying lm_eval/tasks/include/default/Persian/utils.py -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:28:38,581 creating build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:28:38,582 copying lm_eval/tasks/include/default/Arabic/utils.py -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:28:38,584 creating build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:28:38,585 copying lm_eval/tasks/include/default/Ukrainian/utils.py -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:28:38,588 creating build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:28:38,588 copying lm_eval/tasks/include/default/Telugu/utils.py -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:28:38,591 creating build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T14:28:38,591 copying lm_eval/tasks/include/default/Urdu/utils.py -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T14:28:38,594 creating build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:28:38,595 copying lm_eval/tasks/include/default/Lithuanian/utils.py -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:28:38,597 creating build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:28:38,598 copying lm_eval/tasks/include/default/Vietnamese/utils.py -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:28:38,600 creating build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:28:38,601 copying lm_eval/tasks/include/default/Albanian/utils.py -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:28:38,604 creating build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:28:38,604 copying lm_eval/tasks/include/default/Finnish/utils.py -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:28:38,607 creating build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:28:38,608 copying lm_eval/tasks/include/default/Spanish/utils.py -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:28:38,610 creating build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T14:28:38,611 copying lm_eval/tasks/include/default/Nepali/utils.py -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T14:28:38,613 creating build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T14:28:38,614 copying lm_eval/tasks/include/default/Polish/utils.py -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T14:28:38,617 creating build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:28:38,618 copying lm_eval/tasks/include/default/Azerbaijani/utils.py -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:28:38,620 creating build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T14:28:38,621 copying lm_eval/tasks/include/default/Serbian/utils.py -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T14:28:38,623 creating build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:28:38,624 copying lm_eval/tasks/include/default/Portuguese/utils.py -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:28:38,627 creating build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:28:38,628 copying lm_eval/tasks/include/default/French/utils.py -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:28:38,630 creating build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:28:38,631 copying lm_eval/tasks/include/default/Uzbek/utils.py -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:28:38,633 creating build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T14:28:38,634 copying lm_eval/tasks/include/default/Malay/utils.py -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T14:28:38,636 creating build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:28:38,637 copying lm_eval/tasks/include/default/Italian/utils.py -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:28:38,640 creating build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:28:38,641 copying lm_eval/tasks/include/default/Estonian/utils.py -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:28:38,643 creating build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:28:38,644 copying lm_eval/tasks/include/default/Bengali/utils.py -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:28:38,646 creating build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T14:28:38,647 copying lm_eval/tasks/include/default/Georgian/utils.py -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T14:28:38,649 creating build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:28:38,650 copying lm_eval/tasks/include/default/Russian/utils.py -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:28:38,652 creating build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T14:28:38,653 copying lm_eval/tasks/include/default/Korean/utils.py -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T14:28:38,655 creating build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:28:38,656 copying lm_eval/tasks/include/default/Chinese/utils.py -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:28:38,659 creating build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T14:28:38,660 copying lm_eval/tasks/include/default/Croatian/utils.py -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T14:28:38,662 creating build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:28:38,663 copying lm_eval/tasks/include/default/Dutch/utils.py -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:28:38,665 creating build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:28:38,666 copying lm_eval/tasks/include/default/Hindi/utils.py -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:28:38,669 creating build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T14:28:38,670 copying lm_eval/tasks/include/default/Tamil/utils.py -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T14:28:38,672 creating build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:28:38,673 copying lm_eval/tasks/include/default/Hungarian/utils.py -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:28:38,675 creating build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:28:38,676 copying lm_eval/tasks/include/default/Turkish/utils.py -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:28:38,679 creating build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:28:38,679 copying lm_eval/tasks/include/default/Greek/utils.py -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:28:38,682 creating build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:28:38,683 copying lm_eval/tasks/include/default/Armenian/utils.py -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:28:38,685 creating build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T14:28:38,687 copying lm_eval/tasks/include/default/Hebrew/utils.py -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T14:28:38,689 creating build/lib/lm_eval/tasks/include/default/German 2026-05-11T14:28:38,690 copying lm_eval/tasks/include/default/German/utils.py -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T14:28:38,692 creating build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T14:28:38,693 copying lm_eval/tasks/include/default/Japanese/utils.py -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T14:28:38,696 creating build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T14:28:38,697 copying lm_eval/tasks/include/default/Basque/utils.py -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T14:28:38,699 creating build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:28:38,700 copying lm_eval/tasks/include/default/Bulgarian/utils.py -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:28:38,702 creating build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:28:38,703 copying lm_eval/tasks/include/few_shot_og/Indonesian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:28:38,706 creating build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:28:38,706 copying lm_eval/tasks/include/few_shot_og/Tagalog/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:28:38,709 creating build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:28:38,710 copying lm_eval/tasks/include/few_shot_og/North Macedonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:28:38,712 creating build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:28:38,713 copying lm_eval/tasks/include/few_shot_og/Malayalam/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:28:38,715 creating build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:28:38,716 copying lm_eval/tasks/include/few_shot_og/Belarusian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:28:38,718 creating build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:28:38,719 copying lm_eval/tasks/include/few_shot_og/Kazakh/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:28:38,722 creating build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:28:38,723 copying lm_eval/tasks/include/few_shot_og/Persian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:28:38,725 creating build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:28:38,726 copying lm_eval/tasks/include/few_shot_og/Arabic/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:28:38,728 creating build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:28:38,729 copying lm_eval/tasks/include/few_shot_og/Ukrainian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:28:38,731 creating build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:28:38,732 copying lm_eval/tasks/include/few_shot_og/Telugu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:28:38,735 creating build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:28:38,736 copying lm_eval/tasks/include/few_shot_og/Urdu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:28:38,738 creating build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:28:38,739 copying lm_eval/tasks/include/few_shot_og/Lithuanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:28:38,741 creating build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:28:38,742 copying lm_eval/tasks/include/few_shot_og/Vietnamese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:28:38,745 creating build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:28:38,746 copying lm_eval/tasks/include/few_shot_og/Albanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:28:38,748 creating build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:28:38,749 copying lm_eval/tasks/include/few_shot_og/Finnish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:28:38,752 creating build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:28:38,753 copying lm_eval/tasks/include/few_shot_og/Spanish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:28:38,755 creating build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:28:38,756 copying lm_eval/tasks/include/few_shot_og/Nepali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:28:38,758 creating build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:28:38,759 copying lm_eval/tasks/include/few_shot_og/Polish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:28:38,762 creating build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:28:38,763 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:28:38,765 creating build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:28:38,766 copying lm_eval/tasks/include/few_shot_og/Serbian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:28:38,768 creating build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:28:38,769 copying lm_eval/tasks/include/few_shot_og/Portuguese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:28:38,771 creating build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:28:38,772 copying lm_eval/tasks/include/few_shot_og/French/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:28:38,775 creating build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:28:38,776 copying lm_eval/tasks/include/few_shot_og/Uzbek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:28:38,778 creating build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:28:38,779 copying lm_eval/tasks/include/few_shot_og/Malay/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:28:38,781 creating build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:28:38,782 copying lm_eval/tasks/include/few_shot_og/Italian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:28:38,785 creating build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:28:38,786 copying lm_eval/tasks/include/few_shot_og/Estonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:28:38,789 creating build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:28:38,790 copying lm_eval/tasks/include/few_shot_og/Bengali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:28:38,792 creating build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:28:38,794 copying lm_eval/tasks/include/few_shot_og/Georgian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:28:38,796 creating build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:28:38,797 copying lm_eval/tasks/include/few_shot_og/Russian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:28:38,800 creating build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:28:38,801 copying lm_eval/tasks/include/few_shot_og/Korean/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:28:38,803 creating build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:28:38,804 copying lm_eval/tasks/include/few_shot_og/Chinese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:28:38,807 creating build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:28:38,808 copying lm_eval/tasks/include/few_shot_og/Croatian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:28:38,811 creating build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:28:38,812 copying lm_eval/tasks/include/few_shot_og/Dutch/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:28:38,814 creating build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:28:38,815 copying lm_eval/tasks/include/few_shot_og/Hindi/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:28:38,817 creating build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:28:38,818 copying lm_eval/tasks/include/few_shot_og/Tamil/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:28:38,820 creating build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:28:38,821 copying lm_eval/tasks/include/few_shot_og/Hungarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:28:38,824 creating build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:28:38,824 copying lm_eval/tasks/include/few_shot_og/Turkish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:28:38,827 creating build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:28:38,828 copying lm_eval/tasks/include/few_shot_og/Greek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:28:38,830 creating build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:28:38,831 copying lm_eval/tasks/include/few_shot_og/Armenian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:28:38,833 creating build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:28:38,834 copying lm_eval/tasks/include/few_shot_og/Hebrew/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:28:38,836 creating build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:28:38,837 copying lm_eval/tasks/include/few_shot_og/German/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:28:38,840 creating build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:28:38,841 copying lm_eval/tasks/include/few_shot_og/Japanese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:28:38,843 creating build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:28:38,844 copying lm_eval/tasks/include/few_shot_og/Basque/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:28:38,846 creating build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:28:38,847 copying lm_eval/tasks/include/few_shot_og/Bulgarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:28:38,849 creating build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:28:38,851 copying lm_eval/tasks/tmlu/default/utils.py -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:28:38,853 copying lm_eval/tasks/tmlu/default/_generate_configs.py -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:28:38,855 creating build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:28:38,857 copying lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:28:38,859 copying lm_eval/tasks/code_x_glue/code-text/utils.py -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:28:38,862 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:28:38,864 copying lm_eval/tasks/llama3/instruct/mmlu_pt/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:28:38,867 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:28:38,868 copying lm_eval/tasks/llama3/instruct/mmlu_hi/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:28:38,872 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:28:38,874 copying lm_eval/tasks/llama3/instruct/mmlu_de/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:28:38,878 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:28:38,879 copying lm_eval/tasks/llama3/instruct/mmlu_es/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:28:38,882 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:28:38,883 copying lm_eval/tasks/llama3/instruct/mmlu_it/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:28:38,886 creating build/lib/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T14:28:38,887 copying lm_eval/tasks/llama3/instruct/arc_challenge/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T14:28:38,889 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:28:38,890 copying lm_eval/tasks/llama3/instruct/mmlu_fr/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:28:38,894 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:28:38,895 copying lm_eval/tasks/llama3/instruct/mmlu_pro/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:28:38,897 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:28:38,898 copying lm_eval/tasks/llama3/instruct/mmlu_th/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:28:38,901 creating build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T14:28:38,902 copying lm_eval/tasks/leaderboard/musr/utils.py -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T14:28:38,905 creating build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:28:38,906 copying lm_eval/tasks/leaderboard/gpqa/utils.py -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:28:38,909 creating build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:28:38,909 copying lm_eval/tasks/leaderboard/ifeval/instructions_registry.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:28:38,912 copying lm_eval/tasks/leaderboard/ifeval/instructions_util.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:28:38,914 copying lm_eval/tasks/leaderboard/ifeval/instructions.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:28:38,917 copying lm_eval/tasks/leaderboard/ifeval/utils.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:28:38,920 creating build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:28:38,921 copying lm_eval/tasks/leaderboard/math/utils.py -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:28:38,923 creating build/lib/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T14:28:38,924 copying lm_eval/tasks/leaderboard/mmlu_pro/utils.py -> build/lib/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T14:28:38,927 creating build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:28:38,928 copying lm_eval/tasks/basque_bench/flores_eu/create_yamls_flores_eu.py -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:28:38,931 creating build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:28:38,932 copying lm_eval/tasks/galician_bench/flores_gl/create_yamls_flores_gl.py -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:28:38,935 creating build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T14:28:38,936 copying lm_eval/tasks/ifeval/multilingual/instructions_registry.py -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T14:28:38,938 copying lm_eval/tasks/ifeval/multilingual/utils.py -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T14:28:38,941 creating build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:28:38,942 copying lm_eval/tasks/ifeval/multilingual/instructions/ca_instructions.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:28:38,945 copying lm_eval/tasks/ifeval/multilingual/instructions/es_instructions.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:28:38,949 copying lm_eval/tasks/ifeval/multilingual/instructions/__init__.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:28:38,951 creating build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:28:38,952 copying lm_eval/tasks/ifeval/multilingual/instruction_utils/ca_instructions_util.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:28:38,954 copying lm_eval/tasks/ifeval/multilingual/instruction_utils/__init__.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:28:38,956 copying lm_eval/tasks/ifeval/multilingual/instruction_utils/es_instructions_util.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:28:38,959 creating build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T14:28:38,960 copying lm_eval/tasks/aradice/winogrande/metrics.py -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T14:28:38,962 copying lm_eval/tasks/aradice/winogrande/utils.py -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T14:28:38,964 creating build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:28:38,965 copying lm_eval/tasks/aradice/truthfulqa_mcq/metrics.py -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:28:38,968 creating build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:28:38,969 copying lm_eval/tasks/aradice/cultural-benchmark/metrics.py -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:28:38,971 copying lm_eval/tasks/aradice/cultural-benchmark/utils.py -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:28:38,973 creating build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T14:28:38,974 copying lm_eval/tasks/aradice/piqa/metrics.py -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T14:28:38,976 creating build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:28:38,977 copying lm_eval/tasks/aradice/openbookqa/metrics.py -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:28:38,979 copying lm_eval/tasks/aradice/openbookqa/utils.py -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:28:38,982 creating build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:28:38,983 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/metrics.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:28:38,985 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/utils.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:28:38,987 creating build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:28:38,988 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/metrics.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:28:38,990 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/utils.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:28:38,992 creating build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:28:38,994 copying lm_eval/tasks/aradice/boolq/ENG/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:28:38,996 copying lm_eval/tasks/aradice/boolq/ENG/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:28:38,998 creating build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:28:38,999 copying lm_eval/tasks/aradice/boolq/LEV/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:28:39,001 copying lm_eval/tasks/aradice/boolq/LEV/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:28:39,003 creating build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:28:39,004 copying lm_eval/tasks/aradice/boolq/MSA/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:28:39,006 copying lm_eval/tasks/aradice/boolq/MSA/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:28:39,008 creating build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:28:39,009 copying lm_eval/tasks/aradice/boolq/EGY/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:28:39,012 copying lm_eval/tasks/aradice/boolq/EGY/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:28:39,014 creating build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:28:39,015 copying lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:28:39,018 creating build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:28:39,019 copying lm_eval/tasks/okapi/arc_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:28:39,021 creating build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:28:39,022 copying lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:28:39,025 creating build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:28:39,026 copying lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:28:39,029 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:28:39,031 copying lm_eval/tasks/afrixnli/direct/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:28:39,033 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:28:39,034 copying lm_eval/tasks/afrixnli/direct/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:28:39,036 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:28:39,037 copying lm_eval/tasks/afrixnli/direct/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:28:39,039 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:28:39,040 copying lm_eval/tasks/afrixnli/direct/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:28:39,042 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:28:39,043 copying lm_eval/tasks/afrixnli/direct/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:28:39,046 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:28:39,047 copying lm_eval/tasks/afrixnli/translate/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:28:39,050 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:28:39,051 copying lm_eval/tasks/afrixnli/translate/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:28:39,053 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:28:39,054 copying lm_eval/tasks/afrixnli/translate/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:28:39,056 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:28:39,057 copying lm_eval/tasks/afrixnli/translate/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:28:39,060 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:28:39,061 copying lm_eval/tasks/afrixnli/translate/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:28:39,063 creating build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:28:39,065 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/utils.py -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:28:39,067 creating build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:28:39,068 copying lm_eval/tasks/afrixnli/anli prompt/translate/utils.py -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:28:39,070 creating build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:28:39,071 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/utils.py -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:28:39,074 creating build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:28:39,075 copying lm_eval/tasks/afrixnli/lai prompt/direct/utils.py -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:28:39,078 creating build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:28:39,079 copying lm_eval/tasks/afrixnli/lai prompt/translate/utils.py -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:28:39,084 creating build/lib/lm_eval/tasks/click/click_lang 2026-05-11T14:28:39,085 copying lm_eval/tasks/click/click_lang/utils.py -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T14:28:39,088 creating build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:28:39,089 copying lm_eval/tasks/click/click_cul/utils.py -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:28:39,092 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:28:39,093 copying lm_eval/tasks/afrimmlu/direct/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:28:39,096 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:28:39,097 copying lm_eval/tasks/afrimmlu/direct/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:28:39,099 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:28:39,100 copying lm_eval/tasks/afrimmlu/direct/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:28:39,102 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:28:39,103 copying lm_eval/tasks/afrimmlu/direct/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:28:39,105 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:28:39,106 copying lm_eval/tasks/afrimmlu/direct/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:28:39,109 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:28:39,110 copying lm_eval/tasks/afrimmlu/translate/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:28:39,113 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:28:39,114 copying lm_eval/tasks/afrimmlu/translate/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:28:39,116 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:28:39,117 copying lm_eval/tasks/afrimmlu/translate/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:28:39,119 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:28:39,120 copying lm_eval/tasks/afrimmlu/translate/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:28:39,123 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:28:39,124 copying lm_eval/tasks/afrimmlu/translate/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:28:39,126 creating build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:28:39,127 copying lm_eval/tasks/catalan_bench/flores_ca/create_yamls_flores_ca.py -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:28:39,131 creating build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T14:28:39,132 copying lm_eval/tasks/catalan_bench/truthfulqa_va/utils.py -> build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T14:28:39,136 creating build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T14:28:39,137 copying lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T14:28:39,140 creating build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T14:28:39,141 copying lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T14:28:39,143 copying lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T14:28:39,145 creating build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T14:28:39,146 copying lm_eval/tasks/super_glue/cb/aggregate.py -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T14:28:39,148 copying lm_eval/tasks/super_glue/cb/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T14:28:39,150 creating build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T14:28:39,151 copying lm_eval/tasks/super_glue/copa/utils.py -> build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T14:28:39,154 creating build/lib/lm_eval/tasks/super_glue/record 2026-05-11T14:28:39,155 copying lm_eval/tasks/super_glue/record/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T14:28:39,157 copying lm_eval/tasks/super_glue/record/util.py -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T14:28:39,160 creating build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:28:39,162 copying lm_eval/tasks/tmmluplus/default/utils.py -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:28:39,163 copying lm_eval/tasks/tmmluplus/default/_generate_configs.py -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:28:39,177 creating build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:28:39,179 copying lm_eval/tasks/e2lmc/mmlu_early_training/custom_metrics.py -> build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:28:39,182 creating build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:28:39,183 copying lm_eval/tasks/e2lmc/noor/_generate_configs.py -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:28:39,186 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:28:39,188 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:28:39,190 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:28:39,192 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:28:39,194 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:28:39,195 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:28:39,198 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:28:39,201 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:28:39,202 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:28:39,203 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:28:39,206 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:28:39,207 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:28:39,210 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:28:39,211 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:28:39,214 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:28:39,215 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:28:39,218 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:28:39,219 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:28:39,221 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:28:39,223 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:28:39,226 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:28:39,227 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:28:39,229 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:28:39,231 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:28:39,233 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:28:39,235 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:28:39,237 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:28:39,239 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:28:39,241 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:28:39,242 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:28:39,246 creating build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:28:39,248 copying lm_eval/tasks/darija_bench/darija_sentiment/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:28:39,250 creating build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:28:39,251 copying lm_eval/tasks/darija_bench/darija_summarization/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:28:39,254 creating build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:28:39,255 copying lm_eval/tasks/darija_bench/darija_translation/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:28:39,258 creating build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:28:39,259 copying lm_eval/tasks/darija_bench/darija_transliteration/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:28:39,261 creating build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T14:28:39,263 copying lm_eval/tasks/afrobench/naijarc/utils.py -> build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T14:28:39,266 creating build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T14:28:39,267 copying lm_eval/tasks/afrobench/ntrex/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T14:28:39,269 creating build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:28:39,270 copying lm_eval/tasks/afrobench/afrisenti/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:28:39,273 creating build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T14:28:39,274 copying lm_eval/tasks/afrobench/xlsum/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T14:28:39,276 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:28:39,277 copying lm_eval/tasks/afrobench/openai_mmlu/utils.py -> build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:28:39,280 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:28:39,281 copying lm_eval/tasks/afrobench/uhura-arc-easy/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:28:39,283 creating build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:28:39,284 copying lm_eval/tasks/afrobench/masakhapos/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:28:39,286 copying lm_eval/tasks/afrobench/masakhapos/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:28:39,289 creating build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T14:28:39,290 copying lm_eval/tasks/afrobench/flores/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T14:28:39,292 creating build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T14:28:39,293 copying lm_eval/tasks/afrobench/adr/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T14:28:39,296 creating build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T14:28:39,297 copying lm_eval/tasks/afrobench/belebele/utils.py -> build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T14:28:39,300 creating build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T14:28:39,301 copying lm_eval/tasks/afrobench/mafand/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T14:28:39,304 creating build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T14:28:39,305 copying lm_eval/tasks/afrobench/sib/utils.py -> build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T14:28:39,308 creating build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:28:39,308 copying lm_eval/tasks/afrobench/masakhaner/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:28:39,311 creating build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T14:28:39,312 copying lm_eval/tasks/afrobench/afriqa/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T14:28:39,315 creating build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T14:28:39,316 copying lm_eval/tasks/afrobench/injongointent/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T14:28:39,318 creating build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T14:28:39,319 copying lm_eval/tasks/afrobench/salt/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T14:28:39,322 creating build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:28:39,323 copying lm_eval/tasks/afrobench/masakhanews/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:28:39,331 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:28:39,332 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:28:39,334 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:28:39,336 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:28:39,337 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:28:39,339 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:28:39,342 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:28:39,343 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:28:39,344 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:28:39,347 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:28:39,348 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:28:39,349 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:28:39,352 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:28:39,353 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:28:39,355 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:28:39,357 creating build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:28:39,358 copying lm_eval/tasks/afrobench/xlsum/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:28:39,361 creating build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:28:39,361 copying lm_eval/tasks/afrobench/xlsum/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:28:39,364 creating build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:28:39,365 copying lm_eval/tasks/afrobench/xlsum/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:28:39,373 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:28:39,374 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:28:39,376 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:28:39,377 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:28:39,380 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:28:39,381 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:28:39,383 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:28:39,384 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:28:39,386 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:28:39,387 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:28:39,390 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:28:39,391 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:28:39,393 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:28:39,394 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:28:39,397 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:28:39,398 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:28:39,401 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:28:39,402 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:28:39,404 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:28:39,405 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:28:39,416 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:28:39,418 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:28:39,420 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:28:39,421 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:28:39,424 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:28:39,426 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:28:39,428 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:28:39,429 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:28:39,432 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:28:39,434 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:28:39,436 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:28:39,437 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:28:39,440 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:28:39,441 copying lm_eval/tasks/afrobench/sib/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:28:39,443 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:28:39,444 copying lm_eval/tasks/afrobench/sib/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:28:39,447 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:28:39,448 copying lm_eval/tasks/afrobench/sib/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:28:39,450 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:28:39,451 copying lm_eval/tasks/afrobench/sib/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:28:39,453 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:28:39,454 copying lm_eval/tasks/afrobench/sib/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:28:39,457 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:28:39,458 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:28:39,461 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:28:39,462 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:28:39,466 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:28:39,467 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:28:39,470 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:28:39,471 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:28:39,475 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:28:39,476 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:28:39,479 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:28:39,481 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:28:39,484 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:28:39,485 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:28:39,488 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:28:39,490 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:28:39,492 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:28:39,494 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:28:39,497 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:28:39,498 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:28:39,501 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:28:39,502 copying lm_eval/tasks/afrobench/afriqa/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:28:39,505 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:28:39,506 copying lm_eval/tasks/afrobench/afriqa/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:28:39,509 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:28:39,511 copying lm_eval/tasks/afrobench/afriqa/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:28:39,513 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:28:39,515 copying lm_eval/tasks/afrobench/afriqa/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:28:39,518 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:28:39,519 copying lm_eval/tasks/afrobench/afriqa/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:28:39,522 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:28:39,523 copying lm_eval/tasks/afrobench/injongointent/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:28:39,526 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:28:39,527 copying lm_eval/tasks/afrobench/injongointent/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:28:39,530 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:28:39,531 copying lm_eval/tasks/afrobench/injongointent/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:28:39,534 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:28:39,536 copying lm_eval/tasks/afrobench/injongointent/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:28:39,539 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:28:39,540 copying lm_eval/tasks/afrobench/injongointent/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:28:39,545 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:28:39,546 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:28:39,549 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:28:39,550 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:28:39,553 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:28:39,554 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:28:39,558 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:28:39,559 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:28:39,562 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:28:39,563 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:28:39,567 creating build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:28:39,569 copying lm_eval/tasks/spanish_bench/flores_es/create_yamls_flores_es.py -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:28:39,572 creating build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:28:39,573 copying lm_eval/tasks/eq_bench/multilingual/utils.py -> build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:28:39,577 creating build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:28:39,578 copying lm_eval/tasks/score/agi_eval/utils_agieval.py -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:28:39,581 creating build/lib/lm_eval/tasks/score/math 2026-05-11T14:28:39,582 copying lm_eval/tasks/score/math/utils_math.py -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:28:39,584 copying lm_eval/tasks/score/math/math_grader.py -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:28:39,587 creating build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T14:28:39,588 copying lm_eval/tasks/score/mmlu_pro/utils_mmlu_pro.py -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T14:28:39,592 creating build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:28:39,593 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:28:39,598 creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:28:39,599 copying lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:28:39,605 creating build/lib/lm_eval/tasks/noreval/norec 2026-05-11T14:28:39,606 copying lm_eval/tasks/noreval/norec/utils.py -> build/lib/lm_eval/tasks/noreval/norec 2026-05-11T14:28:39,609 creating build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:28:39,610 copying lm_eval/tasks/noreval/norquad/utils.py -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:28:39,613 creating build/lib/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T14:28:39,614 copying lm_eval/tasks/noreval/noropenbookqa/utils.py -> build/lib/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T14:28:39,617 creating build/lib/lm_eval/tasks/noreval/norsumm 2026-05-11T14:28:39,618 copying lm_eval/tasks/noreval/norsumm/utils.py -> build/lib/lm_eval/tasks/noreval/norsumm 2026-05-11T14:28:39,621 creating build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:28:39,622 copying lm_eval/tasks/noreval/ask_gec/errant.py -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:28:39,625 creating build/lib/lm_eval/tasks/noreval/noridiom 2026-05-11T14:28:39,626 copying lm_eval/tasks/noreval/noridiom/utils.py -> build/lib/lm_eval/tasks/noreval/noridiom 2026-05-11T14:28:39,634 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T14:28:39,636 copying lm_eval/tasks/noreval/nortruthfulqa/generation/utils.py -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T14:28:39,638 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:28:39,640 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/utils.py -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:28:39,642 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:28:39,643 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/utils.py -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:28:39,647 creating build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:28:39,649 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/utils.py -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:28:39,651 creating build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:28:39,652 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/utils.py -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:28:39,655 creating build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:28:39,656 copying lm_eval/tasks/mmlusr/question_only/utils.py -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:28:39,659 creating build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:28:39,660 copying lm_eval/tasks/mmlusr/answer_only/utils.py -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:28:39,663 creating build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:28:39,664 copying lm_eval/tasks/mmlusr/question_and_answer/utils.py -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:28:39,675 running egg_info 2026-05-11T14:28:39,685 writing lm_eval.egg-info/PKG-INFO 2026-05-11T14:28:39,711 writing dependency_links to lm_eval.egg-info/dependency_links.txt 2026-05-11T14:28:39,713 writing entry points to lm_eval.egg-info/entry_points.txt 2026-05-11T14:28:39,725 writing requirements to lm_eval.egg-info/requires.txt 2026-05-11T14:28:39,727 writing top-level names to lm_eval.egg-info/top_level.txt 2026-05-11T14:28:42,634 reading manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T14:28:43,380 reading manifest template 'MANIFEST.in' 2026-05-11T14:28:43,380 warning: manifest_maker: MANIFEST.in, line 1: 'recursive-include' expects ... 2026-05-11T14:28:43,381 adding license file 'LICENSE.md' 2026-05-11T14:28:45,123 writing manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T14:28:49,248 copying lm_eval/tasks/glue/mnli/mismatch.yaml -> build/lib/lm_eval/tasks/glue/mnli 2026-05-11T14:28:49,250 copying lm_eval/tasks/glue/mnli/default.yaml -> build/lib/lm_eval/tasks/glue/mnli 2026-05-11T14:28:49,252 creating build/lib/lm_eval/tasks/glue/mrpc 2026-05-11T14:28:49,253 copying lm_eval/tasks/glue/mrpc/default.yaml -> build/lib/lm_eval/tasks/glue/mrpc 2026-05-11T14:28:49,255 creating build/lib/lm_eval/tasks/glue/qnli 2026-05-11T14:28:49,256 copying lm_eval/tasks/glue/qnli/default.yaml -> build/lib/lm_eval/tasks/glue/qnli 2026-05-11T14:28:49,259 creating build/lib/lm_eval/tasks/glue/cola 2026-05-11T14:28:49,260 copying lm_eval/tasks/glue/cola/default.yaml -> build/lib/lm_eval/tasks/glue/cola 2026-05-11T14:28:49,263 creating build/lib/lm_eval/tasks/glue/rte 2026-05-11T14:28:49,264 copying lm_eval/tasks/glue/rte/default.yaml -> build/lib/lm_eval/tasks/glue/rte 2026-05-11T14:28:49,267 creating build/lib/lm_eval/tasks/glue/sst2 2026-05-11T14:28:49,268 copying lm_eval/tasks/glue/sst2/default.yaml -> build/lib/lm_eval/tasks/glue/sst2 2026-05-11T14:28:49,270 creating build/lib/lm_eval/tasks/glue/qqp 2026-05-11T14:28:49,271 copying lm_eval/tasks/glue/qqp/default.yaml -> build/lib/lm_eval/tasks/glue/qqp 2026-05-11T14:28:49,274 creating build/lib/lm_eval/tasks/glue/wnli 2026-05-11T14:28:49,275 copying lm_eval/tasks/glue/wnli/default.yaml -> build/lib/lm_eval/tasks/glue/wnli 2026-05-11T14:28:49,278 copying lm_eval/tasks/nerel-bench/ru_relation_definition.yaml -> build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:28:49,280 copying lm_eval/tasks/nerel-bench/ru_entity_definition.yaml -> build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:28:49,282 copying lm_eval/tasks/nerel-bench/ru_entity_recognition.yaml -> build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:28:49,284 copying lm_eval/tasks/nerel-bench/ru_relation_extraction.yaml -> build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:28:49,287 copying lm_eval/tasks/med_prescriptions/med_prescriptions_easy.yaml -> build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T14:28:49,289 copying lm_eval/tasks/med_prescriptions/med_prescriptions_hard.yaml -> build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T14:28:49,291 creating build/lib/lm_eval/tasks/xnli_eu 2026-05-11T14:28:49,292 copying lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T14:28:49,294 copying lm_eval/tasks/xnli_eu/xnli_eu_native.yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T14:28:49,296 copying lm_eval/tasks/xnli_eu/xnli_eu.yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T14:28:49,298 copying lm_eval/tasks/winogrande/default.yaml -> build/lib/lm_eval/tasks/winogrande 2026-05-11T14:28:49,300 creating build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,301 copying lm_eval/tasks/xstorycloze/default_zh.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,303 copying lm_eval/tasks/xstorycloze/default_eu.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,305 copying lm_eval/tasks/xstorycloze/default_my.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,308 copying lm_eval/tasks/xstorycloze/default_ar.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,310 copying lm_eval/tasks/xstorycloze/default_hi.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,312 copying lm_eval/tasks/xstorycloze/default_ru.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,314 copying lm_eval/tasks/xstorycloze/default_sw.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,317 copying lm_eval/tasks/xstorycloze/default_es.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,319 copying lm_eval/tasks/xstorycloze/_xstorycloze.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,320 copying lm_eval/tasks/xstorycloze/default_te.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,323 copying lm_eval/tasks/xstorycloze/default_en.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,325 copying lm_eval/tasks/xstorycloze/default_id.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:28:49,327 creating build/lib/lm_eval/tasks/haerae 2026-05-11T14:28:49,328 copying lm_eval/tasks/haerae/haerae_sn.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:28:49,330 copying lm_eval/tasks/haerae/haerae_rw.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:28:49,332 copying lm_eval/tasks/haerae/haerae_lw.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:28:49,334 copying lm_eval/tasks/haerae/_haerae.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:28:49,336 copying lm_eval/tasks/haerae/haerae_hi.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:28:49,338 copying lm_eval/tasks/haerae/haerae_gk.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:28:49,340 copying lm_eval/tasks/humaneval/humaneval_plus.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T14:28:49,343 copying lm_eval/tasks/humaneval/humaneval.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T14:28:49,345 copying lm_eval/tasks/humaneval/humaneval_64_instruct.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T14:28:49,347 copying lm_eval/tasks/humaneval/humaneval_64.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T14:28:49,349 copying lm_eval/tasks/humaneval/humaneval_instruct.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T14:28:49,351 creating build/lib/lm_eval/tasks/eus_proficiency 2026-05-11T14:28:49,352 copying lm_eval/tasks/eus_proficiency/eus_proficiency.yaml -> build/lib/lm_eval/tasks/eus_proficiency 2026-05-11T14:28:49,354 creating build/lib/lm_eval/tasks/swag 2026-05-11T14:28:49,355 copying lm_eval/tasks/swag/swag.yaml -> build/lib/lm_eval/tasks/swag 2026-05-11T14:28:49,357 copying lm_eval/tasks/scrolls/scrolls_qmsum.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:49,359 copying lm_eval/tasks/scrolls/scrolls_quality.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:49,361 copying lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:49,363 copying lm_eval/tasks/scrolls/scrolls_qasper.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:49,365 copying lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:49,367 copying lm_eval/tasks/scrolls/scrolls_contractnli.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:49,369 copying lm_eval/tasks/scrolls/scrolls_govreport.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:28:49,371 copying lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias.yaml -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:28:49,373 copying lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias_gen.yaml -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:28:49,375 creating build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,376 copying lm_eval/tasks/multiblimp/multiblimp_bua.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,379 copying lm_eval/tasks/multiblimp/multiblimp_ces.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,381 copying lm_eval/tasks/multiblimp/multiblimp_kaz.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,383 copying lm_eval/tasks/multiblimp/multiblimp_deu.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,385 copying lm_eval/tasks/multiblimp/multiblimp_ron.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,387 copying lm_eval/tasks/multiblimp/multiblimp_chu.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,389 copying lm_eval/tasks/multiblimp/multiblimp_spa.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,391 copying lm_eval/tasks/multiblimp/multiblimp_mkd.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,393 copying lm_eval/tasks/multiblimp/multiblimp_uzb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,395 copying lm_eval/tasks/multiblimp/multiblimp_eus.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,397 copying lm_eval/tasks/multiblimp/multiblimp_sah.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,399 copying lm_eval/tasks/multiblimp/multiblimp_urd.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,401 copying lm_eval/tasks/multiblimp/multiblimp_cym.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,403 copying lm_eval/tasks/multiblimp/multiblimp_guj.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,405 copying lm_eval/tasks/multiblimp/multiblimp_glg.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,407 copying lm_eval/tasks/multiblimp/multiblimp_kxh.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,409 copying lm_eval/tasks/multiblimp/multiblimp_quc.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,411 copying lm_eval/tasks/multiblimp/multiblimp_mdf.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,413 copying lm_eval/tasks/multiblimp/multiblimp_swe.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,415 copying lm_eval/tasks/multiblimp/multiblimp_hit.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,417 copying lm_eval/tasks/multiblimp/multiblimp_abk.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,419 copying lm_eval/tasks/multiblimp/multiblimp_heb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,421 copying lm_eval/tasks/multiblimp/multiblimp_wbp.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,423 copying lm_eval/tasks/multiblimp/multiblimp_kat.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,425 copying lm_eval/tasks/multiblimp/multiblimp_bel.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,427 copying lm_eval/tasks/multiblimp/multiblimp_kmr.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,429 copying lm_eval/tasks/multiblimp/multiblimp_ukr.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,431 copying lm_eval/tasks/multiblimp/multiblimp_tam.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,433 copying lm_eval/tasks/multiblimp/multiblimp_eng.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,435 copying lm_eval/tasks/multiblimp/multiblimp_pol.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,437 copying lm_eval/tasks/multiblimp/multiblimp_azz.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,439 copying lm_eval/tasks/multiblimp/multiblimp_hbs.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,442 copying lm_eval/tasks/multiblimp/multiblimp_hye.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,444 copying lm_eval/tasks/multiblimp/multiblimp_aln.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,445 copying lm_eval/tasks/multiblimp/multiblimp_lav.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,448 copying lm_eval/tasks/multiblimp/multiblimp_got.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,450 copying lm_eval/tasks/multiblimp/multiblimp_slv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,452 copying lm_eval/tasks/multiblimp/multiblimp_arb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,454 copying lm_eval/tasks/multiblimp/multiblimp_por.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,456 copying lm_eval/tasks/multiblimp/multiblimp_fra.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,458 copying lm_eval/tasks/multiblimp/multiblimp_sqi.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,460 copying lm_eval/tasks/multiblimp/multiblimp_vep.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,462 copying lm_eval/tasks/multiblimp/multiblimp_bre.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,464 copying lm_eval/tasks/multiblimp/multiblimp_est.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,466 copying lm_eval/tasks/multiblimp/multiblimp_urb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,468 copying lm_eval/tasks/multiblimp/multiblimp_bho.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,470 copying lm_eval/tasks/multiblimp/multiblimp_xnr.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,471 copying lm_eval/tasks/multiblimp/multiblimp_hbo.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,474 copying lm_eval/tasks/multiblimp/multiblimp_slk.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,476 copying lm_eval/tasks/multiblimp/multiblimp_myv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,478 copying lm_eval/tasks/multiblimp/multiblimp_nds.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,480 copying lm_eval/tasks/multiblimp/multiblimp_fin.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,482 copying lm_eval/tasks/multiblimp/multiblimp_hyw.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,484 copying lm_eval/tasks/multiblimp/multiblimp_bor.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,486 copying lm_eval/tasks/multiblimp/multiblimp_olo.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,488 copying lm_eval/tasks/multiblimp/multiblimp_apu.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,490 copying lm_eval/tasks/multiblimp/multiblimp_hun.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,492 copying lm_eval/tasks/multiblimp/multiblimp_gla.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,494 copying lm_eval/tasks/multiblimp/multiblimp_tur.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,496 copying lm_eval/tasks/multiblimp/multiblimp_fas.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,498 copying lm_eval/tasks/multiblimp/multiblimp_cat.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,500 copying lm_eval/tasks/multiblimp/multiblimp_yrl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,502 copying lm_eval/tasks/multiblimp/multiblimp_rus.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,504 copying lm_eval/tasks/multiblimp/multiblimp_ttc.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,506 copying lm_eval/tasks/multiblimp/multiblimp_ell.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,508 copying lm_eval/tasks/multiblimp/multiblimp_ota.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,511 copying lm_eval/tasks/multiblimp/multiblimp_krl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,513 copying lm_eval/tasks/multiblimp/multiblimp_aqz.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,515 copying lm_eval/tasks/multiblimp/multiblimp_xpg.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,517 copying lm_eval/tasks/multiblimp/multiblimp_dan.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,519 copying lm_eval/tasks/multiblimp/multiblimp_lat.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,521 copying lm_eval/tasks/multiblimp/multiblimp_amh.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,523 copying lm_eval/tasks/multiblimp/multiblimp_fro.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,525 copying lm_eval/tasks/multiblimp/multiblimp_orv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,527 copying lm_eval/tasks/multiblimp/multiblimp_ben.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,529 copying lm_eval/tasks/multiblimp/multiblimp_egy.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,531 copying lm_eval/tasks/multiblimp/multiblimp_xcl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,533 copying lm_eval/tasks/multiblimp/multiblimp_san.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,535 copying lm_eval/tasks/multiblimp/multiblimp_sme.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,537 copying lm_eval/tasks/multiblimp/multiblimp_grc.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,539 copying lm_eval/tasks/multiblimp/multiblimp_fao.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,541 copying lm_eval/tasks/multiblimp/multiblimp_uig.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,542 copying lm_eval/tasks/multiblimp/multiblimp_wol.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,544 copying lm_eval/tasks/multiblimp/multiblimp_frm.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,546 copying lm_eval/tasks/multiblimp/multiblimp_nhi.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,548 copying lm_eval/tasks/multiblimp/multiblimp_pcm.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,550 copying lm_eval/tasks/multiblimp/multiblimp_bul.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,552 copying lm_eval/tasks/multiblimp/multiblimp_hin.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,554 copying lm_eval/tasks/multiblimp/multiblimp_ita.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,556 copying lm_eval/tasks/multiblimp/multiblimp_gle.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,558 copying lm_eval/tasks/multiblimp/multiblimp_mar.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,560 copying lm_eval/tasks/multiblimp/multiblimp_lit.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,562 copying lm_eval/tasks/multiblimp/multiblimp_hsb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,563 copying lm_eval/tasks/multiblimp/multiblimp_isl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,566 copying lm_eval/tasks/multiblimp/multiblimp_kir.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,568 copying lm_eval/tasks/multiblimp/multiblimp_koi.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,570 copying lm_eval/tasks/multiblimp/multiblimp_nld.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,572 copying lm_eval/tasks/multiblimp/multiblimp_kpv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,574 copying lm_eval/tasks/multiblimp/multiblimp_tpn.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,576 copying lm_eval/tasks/multiblimp/multiblimp_lij.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,578 copying lm_eval/tasks/multiblimp/multiblimp_sms.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:28:49,580 copying lm_eval/tasks/glianorex/glianorex_en.yaml -> build/lib/lm_eval/tasks/glianorex 2026-05-11T14:28:49,582 copying lm_eval/tasks/glianorex/glianorex.yaml -> build/lib/lm_eval/tasks/glianorex 2026-05-11T14:28:49,584 copying lm_eval/tasks/glianorex/glianorex_fr.yaml -> build/lib/lm_eval/tasks/glianorex 2026-05-11T14:28:49,586 copying lm_eval/tasks/discrim_eval/discrim_eval_explicit.yaml -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T14:28:49,588 copying lm_eval/tasks/discrim_eval/discrim_eval_implicit.yaml -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T14:28:49,590 copying lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T14:28:49,592 copying lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T14:28:49,594 copying lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T14:28:49,596 copying lm_eval/tasks/race/race.yaml -> build/lib/lm_eval/tasks/race 2026-05-11T14:28:49,599 creating build/lib/lm_eval/tasks/bear 2026-05-11T14:28:49,600 copying lm_eval/tasks/bear/bear_big.yaml -> build/lib/lm_eval/tasks/bear 2026-05-11T14:28:49,602 copying lm_eval/tasks/bear/bear.yaml -> build/lib/lm_eval/tasks/bear 2026-05-11T14:28:49,604 copying lm_eval/tasks/winogender/winogender_male.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:49,606 copying lm_eval/tasks/winogender/winogender_gotcha_male.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:49,608 copying lm_eval/tasks/winogender/winogender.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:49,610 copying lm_eval/tasks/winogender/winogender_neutral.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:49,611 copying lm_eval/tasks/winogender/winogender_female.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:49,613 copying lm_eval/tasks/winogender/winogender_gotcha.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:49,615 copying lm_eval/tasks/winogender/winogender_gotcha_female.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:28:49,618 creating build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,619 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,621 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,623 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,625 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,627 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,629 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,631 copying lm_eval/tasks/bertaqa/bertaqa_en.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,633 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,635 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,637 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,639 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,641 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,643 copying lm_eval/tasks/bertaqa/bertaqa_eu.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,645 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,647 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,649 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:28:49,651 copying lm_eval/tasks/arab_culture/_arab_culture_levant.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,653 copying lm_eval/tasks/arab_culture/arab_culture_uae.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,655 copying lm_eval/tasks/arab_culture/arab_culture_libya.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,657 copying lm_eval/tasks/arab_culture/arab_culture_tunisia.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,659 copying lm_eval/tasks/arab_culture/_arab_culture.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,662 copying lm_eval/tasks/arab_culture/arab_culture_algeria.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,664 copying lm_eval/tasks/arab_culture/arab_culture_syria.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,666 copying lm_eval/tasks/arab_culture/arab_culture_lebanon.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,668 copying lm_eval/tasks/arab_culture/arab_culture_jordan.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,670 copying lm_eval/tasks/arab_culture/arab_culture_egypt.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,672 copying lm_eval/tasks/arab_culture/arab_culture_morocco.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,674 copying lm_eval/tasks/arab_culture/arab_culture_sudan.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,676 copying lm_eval/tasks/arab_culture/_arab_culture_north_africa.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,678 copying lm_eval/tasks/arab_culture/arab_culture_yemen.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,680 copying lm_eval/tasks/arab_culture/_arab_culture_nile_valley.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,682 copying lm_eval/tasks/arab_culture/arab_culture_ksa.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,683 copying lm_eval/tasks/arab_culture/_arab_culture_gulf.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,685 copying lm_eval/tasks/arab_culture/arab_culture_palestine.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:28:49,687 copying lm_eval/tasks/eus_trivia/eus_trivia.yaml -> build/lib/lm_eval/tasks/eus_trivia 2026-05-11T14:28:49,689 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_complete.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T14:28:49,691 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_mt_openbook_qa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:28:49,694 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_leaderboard_arabic_mt_openbook_qa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:28:49,696 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_leaderboard_arabic_mt_piqa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:28:49,698 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_mt_piqa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:28:49,700 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_leaderboard_arabic_mt_boolq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:28:49,702 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_mt_boolq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:28:49,704 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_leaderboard_arabic_mt_copa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:28:49,706 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_mt_copa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:28:49,708 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_mt_arc_challenge.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:28:49,710 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_leaderboard_arabic_mt_arc_challenge.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:28:49,712 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_leaderboard_arabic_exams.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:28:49,714 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_exams.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:28:49,716 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_mt_mmlu.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:28:49,718 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_leaderboard_arabic_mt_mmlu.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:28:49,720 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_mt_arc_easy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:28:49,722 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_leaderboard_arabic_mt_arc_easy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:28:49,724 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,726 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,728 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,730 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,732 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,734 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_marketing.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,736 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,738 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,740 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,742 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,744 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,747 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,749 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,751 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_sociology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,753 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,755 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,757 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,759 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,761 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,763 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,765 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,767 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,769 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,771 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,773 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,775 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,778 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_international_law.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,780 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,782 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,784 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,786 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,788 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,790 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,792 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,794 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,796 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,798 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,800 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,802 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,804 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,807 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,809 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,811 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,813 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,815 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,817 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,819 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,821 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,824 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,826 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,828 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,830 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_management.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,832 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,835 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,837 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,839 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_virology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,841 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,843 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:28:49,846 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,848 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,850 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_mcq_exams_test_ar.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,852 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_sentiment_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,854 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,857 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,859 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_dialects.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,861 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,863 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_msa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,864 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:28:49,867 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_leaderboard_arabic_mt_hellaswag.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:28:49,869 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_mt_hellaswag.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:28:49,871 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_leaderboard_arabic_mt_race.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:28:49,873 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_mt_race.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:28:49,875 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Astronomy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,877 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Architecture.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,879 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mauritania.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,881 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Yemen.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,883 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Comoros.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,885 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromRome.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,887 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Calligraphy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,890 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Morocco.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,892 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromGreece.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,894 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Culture.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,896 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Geography.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,898 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ornament.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,900 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_computer_and_phone.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,902 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Ancient_Egypt.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,904 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Lebanon.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,906 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Sudan.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,908 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Medicine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,910 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Iraq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,912 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromPersia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,914 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Language_Origin.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,916 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Math.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,918 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Philosophy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,920 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ceremony.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,922 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Funeral.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,924 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Clothing.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,926 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_daily_life.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,928 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Tunisia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,930 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_branches_and_schools.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,932 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Somalia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,934 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Algeria.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,936 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_History.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,938 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Palestine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,940 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Egypt_modern.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,942 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_Education.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,944 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Music.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,946 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromIslam.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,948 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Bahrain.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,951 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Wedding.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,953 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mesopotamia_civilization.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,955 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromAncientEgypt.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,957 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromChina.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,959 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Libya.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,961 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arab_Empire.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,963 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Oman.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,966 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Qatar.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,968 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_entertainment.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,970 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,972 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Literature.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,974 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromByzantium.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,976 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_United_Arab_Emirates.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,978 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Saudi_Arabia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,981 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Art.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,983 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Food.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,985 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Syria.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,987 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Kuwait.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,989 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_communication.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,991 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islamic_law_system.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,993 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,996 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Jordan.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:28:49,998 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_leaderboard_arabic_mt_sciq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:28:50,000 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_mt_sciq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:28:50,002 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_leaderboard_arabic_mt_toxigen.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:28:50,004 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_mt_toxigen.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:28:50,006 copying lm_eval/tasks/acpbench/gen_2shot/next_act.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,008 copying lm_eval/tasks/acpbench/gen_2shot/just.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,011 copying lm_eval/tasks/acpbench/gen_2shot/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,013 copying lm_eval/tasks/acpbench/gen_2shot/prog.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,015 copying lm_eval/tasks/acpbench/gen_2shot/land.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,017 copying lm_eval/tasks/acpbench/gen_2shot/reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,019 copying lm_eval/tasks/acpbench/gen_2shot/app.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,021 copying lm_eval/tasks/acpbench/gen_2shot/val.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:28:50,023 creating build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,024 copying lm_eval/tasks/acpbench/mcq_cot_2shot/just.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,027 copying lm_eval/tasks/acpbench/mcq_cot_2shot/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,029 copying lm_eval/tasks/acpbench/mcq_cot_2shot/prog.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,031 copying lm_eval/tasks/acpbench/mcq_cot_2shot/land.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,033 copying lm_eval/tasks/acpbench/mcq_cot_2shot/reach.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,035 copying lm_eval/tasks/acpbench/mcq_cot_2shot/app.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,037 copying lm_eval/tasks/acpbench/mcq_cot_2shot/val.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:28:50,040 creating build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,041 copying lm_eval/tasks/acpbench/boolq_cot_2shot/just.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,043 copying lm_eval/tasks/acpbench/boolq_cot_2shot/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,045 copying lm_eval/tasks/acpbench/boolq_cot_2shot/prog.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,047 copying lm_eval/tasks/acpbench/boolq_cot_2shot/land.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,049 copying lm_eval/tasks/acpbench/boolq_cot_2shot/reach.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,051 copying lm_eval/tasks/acpbench/boolq_cot_2shot/app.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,054 copying lm_eval/tasks/acpbench/boolq_cot_2shot/val.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:28:50,056 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/next_act.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,059 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/just.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,061 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,064 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/prog.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,066 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/land.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,069 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,071 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/app.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,074 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/val.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:28:50,077 copying lm_eval/tasks/jsonschema_bench/jsonschema_bench_easy.yaml -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T14:28:50,080 copying lm_eval/tasks/jsonschema_bench/jsonschema_bench_medium.yaml -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T14:28:50,082 copying lm_eval/tasks/jsonschema_bench/jsonschema_bench_hard.yaml -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T14:28:50,084 copying lm_eval/tasks/eus_reading/eus_reading.yaml -> build/lib/lm_eval/tasks/eus_reading 2026-05-11T14:28:50,086 copying lm_eval/tasks/aclue/aclue_ancient_medical.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,088 copying lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,090 copying lm_eval/tasks/aclue/_aclue.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,092 copying lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,093 copying lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,095 copying lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,097 copying lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,099 copying lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,101 copying lm_eval/tasks/aclue/aclue_couplet_prediction.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,103 copying lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,105 copying lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,107 copying lm_eval/tasks/aclue/aclue_reading_comprehension.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,109 copying lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,111 copying lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,113 copying lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,115 copying lm_eval/tasks/aclue/aclue_ancient_literature.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:28:50,118 copying lm_eval/tasks/careqa/careqa_en.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:50,121 copying lm_eval/tasks/careqa/careqa_es.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:50,123 copying lm_eval/tasks/careqa/careqa_open.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:50,125 copying lm_eval/tasks/careqa/careqa_open_perplexity.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:28:50,127 copying lm_eval/tasks/icelandic_winogrande/default.yaml -> build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T14:28:50,129 copying lm_eval/tasks/hrm8k/en/hrm8k_gsm8k_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:50,131 copying lm_eval/tasks/hrm8k/en/hrm8k_mmmlu_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:50,133 copying lm_eval/tasks/hrm8k/en/hrm8k_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:50,135 copying lm_eval/tasks/hrm8k/en/hrm8k_math_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:50,137 copying lm_eval/tasks/hrm8k/en/hrm8k_omni_math_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:50,139 copying lm_eval/tasks/hrm8k/en/hrm8k_ksm_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:28:50,141 copying lm_eval/tasks/hrm8k/default/hrm8k_gsm8k.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:50,143 copying lm_eval/tasks/hrm8k/default/hrm8k_omni_math.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:50,145 copying lm_eval/tasks/hrm8k/default/hrm8k.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:50,148 copying lm_eval/tasks/hrm8k/default/hrm8k_mmmlu.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:50,149 copying lm_eval/tasks/hrm8k/default/hrm8k_math.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:50,151 copying lm_eval/tasks/hrm8k/default/hrm8k_ksm.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:28:50,153 copying lm_eval/tasks/copal_id/colloquial.yaml -> build/lib/lm_eval/tasks/copal_id 2026-05-11T14:28:50,155 copying lm_eval/tasks/copal_id/standard.yaml -> build/lib/lm_eval/tasks/copal_id 2026-05-11T14:28:50,157 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_libya.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,159 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_algeria.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,161 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_palestine.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,163 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_egypt.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,165 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_lebanon.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,167 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_gulf.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,169 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,171 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_uae.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,173 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_yemen.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,176 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_tunisia.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,178 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_morocco.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,180 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_sudan.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,182 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_syria.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,184 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_ksa.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,186 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_north_africa.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,188 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_jordan.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,190 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_nile_valley.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,192 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_levant.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:28:50,194 creating build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,195 copying lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,197 copying lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,199 copying lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,201 copying lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,203 copying lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,205 copying lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,207 copying lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,209 copying lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,211 copying lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,212 copying lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:28:50,214 copying lm_eval/tasks/medtext/medtext.yaml -> build/lib/lm_eval/tasks/medtext 2026-05-11T14:28:50,216 copying lm_eval/tasks/medtext/medtext_perplexity.yaml -> build/lib/lm_eval/tasks/medtext 2026-05-11T14:28:50,218 copying lm_eval/tasks/pisa/pisa_en.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,220 copying lm_eval/tasks/pisa/pisa_fr.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,222 copying lm_eval/tasks/pisa/pisa_ch_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,224 copying lm_eval/tasks/pisa/pisa_ch.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,226 copying lm_eval/tasks/pisa/pisa_de.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,228 copying lm_eval/tasks/pisa/pisa_es_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,230 copying lm_eval/tasks/pisa/pisa_fr_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,232 copying lm_eval/tasks/pisa/pisa_it_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,234 copying lm_eval/tasks/pisa/pisa_es.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,236 copying lm_eval/tasks/pisa/pisa_it.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,238 copying lm_eval/tasks/pisa/pisa_de_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,240 copying lm_eval/tasks/pisa/_pisa.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,241 copying lm_eval/tasks/pisa/_pisa_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,244 copying lm_eval/tasks/pisa/pisa_en_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:28:50,246 copying lm_eval/tasks/xnli/xnli_sw.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,248 copying lm_eval/tasks/xnli/xnli_bg.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,250 copying lm_eval/tasks/xnli/xnli_zh.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,252 copying lm_eval/tasks/xnli/xnli_ur.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,254 copying lm_eval/tasks/xnli/xnli_th.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,256 copying lm_eval/tasks/xnli/xnli_de.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,258 copying lm_eval/tasks/xnli/xnli_es.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,260 copying lm_eval/tasks/xnli/_xnli.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,262 copying lm_eval/tasks/xnli/xnli_ru.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,264 copying lm_eval/tasks/xnli/xnli_tr.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,266 copying lm_eval/tasks/xnli/xnli_el.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,268 copying lm_eval/tasks/xnli/xnli_en.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,270 copying lm_eval/tasks/xnli/xnli_ar.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,272 copying lm_eval/tasks/xnli/xnli_fr.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,274 copying lm_eval/tasks/xnli/xnli_vi.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,276 copying lm_eval/tasks/xnli/xnli_hi.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:28:50,278 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,280 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,282 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,284 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,286 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,288 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,290 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,292 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,294 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,296 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,298 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,300 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,302 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,304 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,306 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,308 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,310 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,312 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,314 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,316 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,318 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,321 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,323 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,325 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,327 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,329 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,331 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,333 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,335 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,337 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,339 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,341 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,343 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,345 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,347 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,349 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,351 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,353 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,354 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,356 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,359 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,361 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,363 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,365 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,367 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,369 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,371 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,374 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,377 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,380 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,382 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,384 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,386 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,388 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,391 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,393 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,395 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,397 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,399 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,401 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,403 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,405 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,407 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,409 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,411 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,413 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,415 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,417 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,419 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,421 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,423 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,425 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,428 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,430 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,433 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,435 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,438 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,440 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,444 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,446 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,448 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,450 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,452 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,454 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,456 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,458 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,460 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,462 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,465 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,467 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,469 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,472 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,474 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,476 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,478 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,481 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,483 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,487 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,489 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,492 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,494 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,496 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,498 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,501 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,503 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,505 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,508 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,510 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,512 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,515 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,517 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,519 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,521 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,523 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,526 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,528 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,530 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,532 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,534 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,536 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,538 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,541 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,543 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,545 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,547 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,550 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,552 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,554 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,557 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,560 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,562 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,564 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,566 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,568 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,571 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,573 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,575 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,577 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,579 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,581 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,583 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,586 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,588 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,590 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,592 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,595 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,597 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,599 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,601 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,603 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,605 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,608 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,609 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,612 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,614 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,616 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,618 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,620 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,622 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,624 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,626 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,628 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,631 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,633 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,635 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,637 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,639 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,641 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,643 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,645 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,647 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,649 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,651 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,653 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,655 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,657 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,659 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,661 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,664 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,666 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,669 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,672 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,674 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,676 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,678 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,680 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,682 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,684 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,686 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,689 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,691 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,693 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,695 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,697 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,699 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,702 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,704 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,706 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,708 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,710 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,712 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,714 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,716 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,718 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,720 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,722 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,724 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,726 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,728 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,731 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,733 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,735 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,737 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,740 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,742 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,744 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,746 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,748 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,750 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,752 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,754 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,756 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,759 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,761 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,763 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,765 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,767 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,769 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,771 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,773 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,775 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,778 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,780 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,782 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,784 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,787 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,789 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,791 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,793 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,795 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,797 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,800 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,802 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,805 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,808 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,811 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,814 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,816 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,818 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,820 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,822 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,824 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,826 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,829 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,831 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,833 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,835 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,837 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,839 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,841 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,843 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,845 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,848 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,850 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,852 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,854 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,856 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,858 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,860 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,862 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,864 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,867 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,869 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,871 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,873 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,875 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,877 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,879 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,881 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,883 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,885 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,887 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,889 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,891 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,893 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,895 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,897 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,899 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,901 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,903 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,905 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,907 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,909 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,911 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,913 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,916 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,918 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,921 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,923 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,925 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,927 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,929 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,931 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,933 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,935 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,937 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,939 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,941 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,943 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,945 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,948 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,950 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,952 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,954 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,956 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,958 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,960 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,963 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,965 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,967 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,969 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,971 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,973 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,975 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,977 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,979 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,981 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,983 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,985 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,987 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,989 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,991 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,993 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,995 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,997 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:50,999 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,002 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,004 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,006 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,008 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,010 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,012 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,014 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,016 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,018 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,020 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,022 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,024 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,026 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,029 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,031 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,033 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,035 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,037 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,039 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,041 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,043 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,045 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,047 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,049 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,051 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,053 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,056 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,058 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,060 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,062 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,064 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,067 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,069 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,071 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,073 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,075 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,077 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,079 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,081 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,084 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,086 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,088 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,090 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,092 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,095 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,097 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,099 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,102 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,104 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,106 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,109 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,111 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,113 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,115 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,117 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,119 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,121 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,123 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,125 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,127 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,129 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,131 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,133 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,135 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,137 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,139 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,141 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,143 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,146 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,148 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,150 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,152 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,154 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,156 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,158 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,160 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,163 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,165 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,168 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,170 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,172 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,174 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,176 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,178 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,180 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,182 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,184 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,186 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,188 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,190 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,192 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,194 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,196 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,198 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,200 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,202 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,205 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,207 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,209 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,211 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,213 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,214 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,216 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,218 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,220 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,222 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,224 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,226 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,228 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,230 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,232 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,234 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,236 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,238 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,241 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,243 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,245 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,247 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,249 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,251 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,253 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,255 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,257 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,259 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,261 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,263 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,265 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,267 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,269 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,271 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,273 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,275 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,276 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,278 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,280 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,282 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,284 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,286 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,288 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,290 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,292 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,294 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,296 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,299 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,301 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,303 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,305 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,307 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,309 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,311 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,313 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,315 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,318 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,320 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,322 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,324 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,326 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,328 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,330 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,332 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,334 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,336 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,338 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,340 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,342 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,344 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,346 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,349 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,351 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,353 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,355 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,357 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,359 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,361 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,363 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,366 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,368 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,370 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,372 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,374 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,377 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,379 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,381 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,382 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,384 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,386 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,388 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,390 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,392 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,394 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,396 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,398 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,400 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,402 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,404 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,407 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,409 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,411 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,413 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,415 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,417 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,419 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,421 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,423 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,425 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,428 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,430 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,432 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,434 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,436 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,438 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,440 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,441 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,443 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,445 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,447 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,449 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,451 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,453 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,455 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,458 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,460 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,462 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,464 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,466 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,468 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,470 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,472 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,474 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,476 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,478 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,480 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,482 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,485 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,487 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,489 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,490 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,492 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,494 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,496 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,498 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,500 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,502 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,504 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,506 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,508 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,510 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,512 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,514 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,516 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,518 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,520 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,522 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,524 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,527 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,529 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,531 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,533 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,535 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,537 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,539 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,541 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,543 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,545 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,547 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,549 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,551 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,553 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,555 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,557 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,559 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,561 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,563 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,565 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,567 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,569 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,571 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,573 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,575 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,577 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,579 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,581 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,583 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,585 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,587 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,589 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,592 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,594 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,596 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,598 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,600 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,602 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,604 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,606 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,608 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,610 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,612 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,614 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,616 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,618 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,620 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,622 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,624 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,626 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,628 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,630 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,632 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,634 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,636 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,638 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,640 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,642 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,645 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,647 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,649 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,651 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,653 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,655 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,657 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,659 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,661 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,663 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,665 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,667 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,669 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,672 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,675 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,677 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,679 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,682 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,684 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,687 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,689 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,691 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,693 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,695 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,698 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,701 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,704 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,706 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,709 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,711 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,715 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,718 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,720 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,722 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,725 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,727 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,730 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,732 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,735 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,737 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,740 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,742 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,745 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,748 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,750 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,753 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,755 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,758 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,760 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,763 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,765 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,768 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,771 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,773 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,776 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,778 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,781 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,784 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,787 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,790 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,793 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,796 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,798 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,801 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,803 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,805 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,808 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,810 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,813 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,815 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,817 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,820 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,822 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,824 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,827 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,830 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,832 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,835 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,837 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,839 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,842 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,844 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,847 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,849 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,852 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,854 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,856 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,859 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,861 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,864 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,866 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,868 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,870 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,872 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,874 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,876 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,878 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,880 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,882 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,884 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,886 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,888 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,890 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,892 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,894 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,896 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,898 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,900 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,903 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,905 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,907 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,909 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,911 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,913 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,915 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,917 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,919 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,921 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,923 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,925 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,927 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,929 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,931 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,933 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,935 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,938 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,940 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,942 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,944 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,946 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,948 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,950 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,952 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,954 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,956 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,958 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,960 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,963 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,965 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,967 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,969 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,971 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,972 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,975 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,977 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,979 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,981 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,983 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,985 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,988 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,990 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,992 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,994 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,996 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:51,998 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,000 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,002 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,004 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,006 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,008 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,010 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,013 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,015 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,017 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,019 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,022 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,024 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,026 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,028 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,030 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,032 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,034 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,036 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,038 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,040 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,042 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,044 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,047 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,049 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,052 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,054 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,056 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,058 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,060 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,062 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,064 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,066 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,068 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,071 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,073 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,075 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,077 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,079 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,081 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,083 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,085 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,088 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,090 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,092 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,094 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,096 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,098 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,100 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,102 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,104 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,106 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,108 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,110 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,112 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,114 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,116 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,119 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,121 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,123 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,125 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,127 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,129 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,131 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,133 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,135 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,137 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:28:52,139 copying lm_eval/tasks/meqsum/meqsum.yaml -> build/lib/lm_eval/tasks/meqsum 2026-05-11T14:28:52,141 creating build/lib/lm_eval/tasks/storycloze 2026-05-11T14:28:52,142 copying lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/lib/lm_eval/tasks/storycloze 2026-05-11T14:28:52,144 copying lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/lib/lm_eval/tasks/storycloze 2026-05-11T14:28:52,146 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,148 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,150 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,152 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,154 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,157 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,159 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,161 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,163 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,165 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,167 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,169 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,171 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,173 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,175 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,178 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,180 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,182 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,184 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,186 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,189 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,192 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,194 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,196 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,198 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,200 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,202 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,204 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,206 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,208 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,210 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,213 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,215 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,217 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,219 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,222 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,224 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,226 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,228 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,230 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,232 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,234 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,237 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,239 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,241 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,244 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,246 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,248 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,251 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,253 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,255 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,258 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,260 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,262 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,264 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,266 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,269 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,271 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,273 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,275 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,277 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,279 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:28:52,281 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,283 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,285 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,288 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,290 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,292 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,294 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,296 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,298 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,300 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,302 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,304 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,307 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,308 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,310 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,312 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,314 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,316 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,318 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,320 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,322 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,324 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,326 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,328 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,330 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,332 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,335 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,337 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,339 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,341 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,343 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,345 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,348 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,350 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,352 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,354 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,356 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,358 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,361 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,363 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,364 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,367 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,369 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,371 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,373 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,375 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,377 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,379 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,382 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,384 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,386 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,388 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,390 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,392 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,395 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,398 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,400 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,402 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,404 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,406 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,408 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,411 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:28:52,413 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,415 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,417 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,419 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,421 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,423 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,426 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,428 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,430 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,432 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,434 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,436 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,438 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,440 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,442 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,444 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,446 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,448 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,450 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,452 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,455 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,457 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,461 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,463 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,465 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,467 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,470 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,472 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,474 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,476 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,478 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,480 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,482 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,484 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,486 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,488 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,490 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,493 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,495 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,497 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,499 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,501 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,503 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,506 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,508 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,511 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,514 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,516 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,520 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,523 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,526 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,529 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,531 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,534 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,537 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,539 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,542 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,545 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,548 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,550 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,553 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,556 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:28:52,558 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,561 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,564 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,567 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,569 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,572 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,575 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,577 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,580 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,582 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,585 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,588 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,591 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,593 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,596 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,598 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,601 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,604 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,606 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,608 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,611 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,613 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,615 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,617 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,619 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,620 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,622 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,624 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,626 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,628 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,631 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,633 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,634 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,637 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,639 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,641 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,643 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,645 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,647 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,649 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,651 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,653 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,655 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,657 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,659 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,661 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,663 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,665 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,666 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,668 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,670 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,672 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,674 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,676 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,678 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,680 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,682 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,684 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,686 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,688 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,690 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,692 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:28:52,694 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,696 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,698 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,701 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,703 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,705 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,707 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,709 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,711 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,713 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,715 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,717 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,719 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,721 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,723 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,725 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,727 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,729 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,731 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,733 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,735 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,737 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,740 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,742 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,744 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,746 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,748 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,750 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,752 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,754 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,756 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,758 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,760 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,762 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,764 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,766 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,768 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,770 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,772 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,774 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,776 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,778 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,780 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,782 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,784 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,786 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,788 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,790 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,792 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,795 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,797 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,799 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,801 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,803 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,805 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,808 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,810 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,812 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,814 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,816 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,818 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,820 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:28:52,822 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,824 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,826 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,828 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,830 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,832 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,833 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,836 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,838 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,840 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,842 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,844 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,846 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,849 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,851 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,854 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,856 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,858 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,860 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,862 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,864 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,866 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,868 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,871 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,873 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,876 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,878 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,880 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,882 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,884 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,886 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,888 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,890 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,892 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,894 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,896 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,898 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,900 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,902 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,904 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,906 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,909 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,911 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,913 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,915 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,917 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,919 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,921 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,923 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,925 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,927 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,929 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,932 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,934 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,936 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,938 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,940 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,942 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,944 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,946 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,948 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,950 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:28:52,952 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,955 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,957 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,959 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,961 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,963 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,965 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,967 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,969 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,971 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,973 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,976 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,978 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,980 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,982 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,984 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,986 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,988 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,990 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,992 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,994 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,996 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:52,998 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,000 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,002 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,004 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,006 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,008 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,011 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,013 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,015 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,017 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,019 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,021 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,023 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,025 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,027 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,029 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,031 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,034 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,036 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,038 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,040 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,042 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,044 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,046 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,048 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,050 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,052 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,054 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,056 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,058 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,060 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,062 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,064 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,066 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,068 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,070 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,072 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,074 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,077 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,079 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:28:53,081 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,083 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,085 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,087 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,089 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,091 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,093 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,095 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,097 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,099 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,101 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,103 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,105 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,107 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,110 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,112 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,114 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,116 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,118 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,120 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,122 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,124 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,126 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,129 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,131 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,133 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,135 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,138 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,140 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,142 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,144 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,146 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,148 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,150 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,152 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,154 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,156 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,158 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,160 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,163 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,165 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,167 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,169 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,171 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,173 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,175 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,178 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,180 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,182 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,184 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,186 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,188 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,190 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,192 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,194 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,196 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,198 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,200 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,202 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,204 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,206 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,208 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:28:53,210 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,212 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,214 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,216 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,218 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,220 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,222 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,224 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,226 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,228 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,230 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,232 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,234 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,236 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,238 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,240 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,242 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,244 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,246 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,248 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,250 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,252 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,254 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,256 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,257 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,259 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,261 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,263 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,265 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,267 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,269 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,271 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,273 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,275 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,277 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,279 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,281 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,283 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,285 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,287 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,289 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,291 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,293 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,295 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,297 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,299 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,301 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,303 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,305 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,307 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,309 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,311 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,312 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,314 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,316 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,318 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,320 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,322 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,324 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,326 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,328 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,330 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:28:53,332 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,334 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,336 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,338 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,340 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,342 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,344 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,346 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,348 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,350 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,352 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,354 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,356 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,358 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,359 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,361 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,363 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,365 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,367 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,369 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,371 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,373 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,375 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,377 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,379 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,381 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,383 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,385 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,387 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,389 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,391 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,393 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,395 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,398 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,400 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,402 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,405 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,407 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,409 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,411 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,413 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,415 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,417 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,420 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,422 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,424 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,426 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,428 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,430 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,432 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,434 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,436 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,439 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,441 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,443 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,446 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,448 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,450 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,452 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,455 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,457 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,459 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:28:53,462 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,465 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,467 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,469 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,471 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,473 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,475 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,477 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,479 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,481 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,483 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,486 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,488 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,490 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,492 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,494 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,496 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,498 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,501 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,503 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,505 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,507 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,509 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,511 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,513 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,515 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,517 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,519 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,521 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,523 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,525 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,527 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,529 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,531 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,533 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,535 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,537 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,540 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,542 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,544 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,546 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,549 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,551 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,553 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,555 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,557 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,559 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,562 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,564 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,566 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,568 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,570 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,572 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,574 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,576 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,579 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,581 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,583 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,585 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,587 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,589 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,591 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:28:53,593 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,595 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,598 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,600 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,602 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,604 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,606 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,608 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,610 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,612 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,614 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,616 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,618 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,620 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,623 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,625 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,627 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,629 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,631 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,633 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,635 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,637 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,639 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,641 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,643 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,646 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,648 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,650 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,652 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,654 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,656 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,658 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,660 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,663 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,665 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,667 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,669 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,671 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,673 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,675 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,677 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,680 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,682 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,684 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,686 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,688 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,690 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,692 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,694 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,696 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,698 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,700 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,703 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,705 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,707 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,709 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,711 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,713 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,715 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,717 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,719 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,721 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:28:53,723 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,726 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,728 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,730 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,732 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,734 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,737 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,739 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,740 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,742 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,745 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,747 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,749 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,751 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,754 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,756 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,758 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,760 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,762 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,764 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,766 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,768 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,770 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,772 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,775 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,777 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,779 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,781 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,783 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,785 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,788 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,790 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,792 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,794 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,796 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,798 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,800 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,802 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,804 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,806 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,808 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,810 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,812 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,814 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,816 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,819 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,821 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,823 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,825 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,827 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,829 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,831 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,834 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,836 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,839 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,841 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,843 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,845 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,848 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,850 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,852 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,854 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:28:53,856 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,858 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,860 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,862 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,864 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,866 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,868 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,870 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,873 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,875 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,877 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,879 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,882 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,884 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,886 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,888 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,890 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,892 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,894 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,896 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,898 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,900 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,903 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,905 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,908 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,911 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,913 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,915 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,917 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,919 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,921 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,923 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,925 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,927 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,929 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,931 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,933 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,935 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,938 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,940 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,942 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,944 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,946 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,948 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,950 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,952 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,955 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,957 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,959 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,961 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,962 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,964 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,966 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,968 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,970 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,972 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,975 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,977 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,979 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,982 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,985 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,988 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:28:53,990 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:53,992 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:53,994 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:53,997 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:53,999 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,001 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,003 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,005 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,007 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,009 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,011 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,013 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,015 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,018 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,020 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,022 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,024 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,026 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,029 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,031 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,033 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,035 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,037 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,039 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,042 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,044 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,047 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,049 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,051 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,053 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,055 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,057 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,060 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,063 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,065 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,068 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,070 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,073 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,075 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,077 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,079 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,081 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,083 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,085 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,087 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,089 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,091 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,093 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,095 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,098 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,100 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,103 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,106 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,108 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,110 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,112 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,114 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,117 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,119 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,121 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,123 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,125 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:28:54,128 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,130 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,132 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,135 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,137 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,138 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,140 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,142 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,144 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,147 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,149 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,151 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,153 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,155 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,157 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,159 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,161 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,163 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,165 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,168 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,170 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,172 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,175 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,177 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,179 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,181 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,184 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,186 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,188 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,189 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,192 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,194 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,195 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,197 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,199 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,201 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,203 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,205 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,207 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,209 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,211 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,213 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,215 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,217 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,219 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,221 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,223 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,225 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,227 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,229 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,231 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,233 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,235 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,237 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,239 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,241 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,243 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,245 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,247 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,249 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,251 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,253 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:28:54,255 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,257 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,260 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,262 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,264 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,266 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,268 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,270 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,272 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,274 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,276 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,278 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,280 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,282 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,284 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,286 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,288 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,290 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,292 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,294 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,296 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,298 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,300 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,302 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,304 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,306 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,308 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,310 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,312 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,314 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,316 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,318 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,320 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,322 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,324 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,327 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,329 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,331 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,333 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,335 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,337 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,339 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,341 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,343 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,345 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,347 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,349 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,351 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,353 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,355 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,357 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,359 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,361 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,363 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,365 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,367 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,369 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,371 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,373 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,375 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,377 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,379 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:28:54,381 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,383 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,385 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,387 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,389 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,391 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,393 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,395 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,398 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,399 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,401 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,403 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,405 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,407 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,409 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,411 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,413 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,415 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,417 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,420 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,422 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,424 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,426 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,428 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,430 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,432 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,435 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,437 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,439 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,441 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,443 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,445 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,448 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,450 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,452 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,454 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,456 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,458 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,460 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,462 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,464 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,466 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,468 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,470 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,472 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,474 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,476 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,478 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,480 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,482 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,485 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,487 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,489 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,491 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,493 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,495 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,497 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,499 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,501 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,503 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,505 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,507 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:28:54,509 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,511 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,513 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,515 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,517 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,519 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,521 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,523 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,525 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,527 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,529 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,531 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,533 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,535 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,537 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,539 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,541 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,543 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,545 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,547 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,549 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,551 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,553 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,555 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,557 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,559 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,561 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,563 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,564 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,566 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,568 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,570 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,572 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,574 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,576 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,579 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,581 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,582 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,584 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,586 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,588 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,590 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,593 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,596 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,598 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,600 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,603 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,605 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,608 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,610 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,613 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,615 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,617 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,619 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,621 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,623 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,625 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,627 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,629 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,631 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,633 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,635 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:28:54,638 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,640 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,642 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,644 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,646 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,648 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,650 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,652 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,654 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,656 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,658 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,660 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,662 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,664 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,667 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,669 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,671 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,673 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,674 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,677 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,679 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,681 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,683 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,685 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,687 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,689 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,691 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,693 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,695 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,698 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,700 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,702 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,704 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,706 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,708 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,710 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,712 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,714 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,716 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,718 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,720 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,722 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,724 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,726 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,728 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,730 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,732 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,734 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,736 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,738 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,740 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,742 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,744 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,745 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,748 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,750 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,752 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,754 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,756 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,758 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,760 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,762 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:28:54,764 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,766 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,768 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,770 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,772 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,774 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,776 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,778 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,780 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,782 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,784 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,786 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,788 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,790 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,792 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,794 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,796 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,798 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,800 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,802 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,804 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,806 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,808 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,810 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,812 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,814 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,816 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,818 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,820 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,823 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,825 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,827 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,829 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,831 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,833 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,835 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,837 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,839 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,840 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,842 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,844 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,847 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,849 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,851 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,853 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,855 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,857 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,859 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,861 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,863 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,865 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,867 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,869 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,871 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,873 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,875 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,878 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,880 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,882 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,884 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,887 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,889 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:28:54,891 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,893 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,895 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,897 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,899 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,901 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,903 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,905 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,907 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,910 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,912 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,914 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,916 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,918 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,920 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,922 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,924 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,926 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,928 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,930 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,933 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,934 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,936 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,938 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,940 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,942 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,944 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,947 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,949 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,950 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,952 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,954 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,956 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,958 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,960 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,962 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,964 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,966 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,970 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,972 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,974 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,976 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,979 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,981 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,983 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,985 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,987 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,989 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,991 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,993 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,994 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,997 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:54,999 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,001 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,003 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,005 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,007 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,009 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,011 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,013 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,015 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,017 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:28:55,019 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,021 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,023 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,025 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,027 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,029 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,031 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,033 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,035 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,037 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,039 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,041 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,043 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,045 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,047 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,049 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,051 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,053 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,055 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,057 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,059 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,061 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,063 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,065 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,067 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,069 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,071 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,073 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,075 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,077 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,079 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,081 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,083 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,085 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,088 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,090 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,092 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,094 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,097 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,099 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,101 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,103 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,105 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,107 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,109 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,111 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,113 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,115 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,117 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,119 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,121 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,123 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,126 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,128 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,131 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,133 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,135 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,138 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,140 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,142 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,145 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,147 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:28:55,149 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,151 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,153 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,155 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,157 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,159 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,162 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,164 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,166 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,169 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,171 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,174 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,176 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,178 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,180 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,182 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,184 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,186 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,188 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,190 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,192 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,195 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,197 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,199 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,201 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,203 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,205 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,208 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,210 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,212 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,214 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,217 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,219 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,221 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,223 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,225 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,227 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,229 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,231 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,233 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,235 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,237 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,239 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,241 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,243 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,245 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,247 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,249 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,251 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,254 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,256 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,258 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,260 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,262 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,264 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,266 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,268 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,270 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,272 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,274 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,276 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,278 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:28:55,280 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,282 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,284 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,286 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,288 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,290 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,292 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,294 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,296 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,299 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,301 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,303 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,305 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,307 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,310 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,312 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,314 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,316 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,318 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,320 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,322 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,324 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,326 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,328 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,330 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,332 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,334 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,336 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,338 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,341 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,343 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,345 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,347 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,349 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,351 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,353 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,356 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,358 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,360 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,362 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,364 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,366 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,368 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,370 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,372 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,374 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,376 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,378 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,381 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,382 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,384 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,386 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,388 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,391 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,393 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,395 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,397 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,399 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,401 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,403 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,405 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,407 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:28:55,409 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,411 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,414 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,416 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,418 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,421 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,424 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,427 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,429 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,431 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,433 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,435 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,437 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,439 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,441 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,443 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,445 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,448 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,450 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,452 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,454 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,456 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,458 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,460 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,462 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,465 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,467 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,469 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,471 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,473 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,475 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,477 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,480 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,482 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,484 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,486 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,488 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,490 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,492 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,495 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,497 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,499 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,501 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,503 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,506 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,508 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,510 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,513 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,515 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,517 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,519 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,521 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,523 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,525 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,527 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,529 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,532 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,534 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,536 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,538 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,540 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,542 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:28:55,544 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,546 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,548 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,550 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,552 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,554 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,557 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,559 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,562 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,565 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,567 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,569 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,572 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,574 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,576 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,578 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,581 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,583 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,585 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,587 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,589 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,591 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,594 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,596 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,598 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,600 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,602 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,604 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,606 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,608 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,610 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,612 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,615 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,617 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,619 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,621 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,623 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,625 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,627 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,630 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,632 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,634 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,636 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,638 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,641 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,643 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,645 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,647 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,649 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,651 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,655 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,657 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,659 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,661 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,663 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,665 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,667 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,669 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,671 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,673 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,675 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,678 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:28:55,680 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,682 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,684 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,686 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,688 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,690 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,692 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,694 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,697 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,699 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,701 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,704 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,706 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,708 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,710 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,712 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,714 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,716 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,718 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,720 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,722 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,724 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,726 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,728 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,730 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,732 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,735 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,737 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,739 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,741 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,743 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,745 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,747 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,749 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,751 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,753 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,756 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,758 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,760 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,762 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,764 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,766 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,768 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,770 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,772 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,774 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,776 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,778 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,780 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,783 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,785 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,787 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,790 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,792 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,795 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,797 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,799 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,801 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,804 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,806 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,808 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,810 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:28:55,812 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,814 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,816 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,818 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,820 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,822 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,824 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,826 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,828 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,830 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,832 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,834 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,836 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,838 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,840 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,842 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,844 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,847 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,849 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,851 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,853 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,854 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,857 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,859 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,861 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,863 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,865 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,867 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,869 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,871 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,873 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,875 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,877 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,879 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,881 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,883 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,885 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,886 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,889 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,891 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,893 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,895 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,897 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,899 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,901 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,903 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,905 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,907 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,909 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,911 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,913 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,915 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,917 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,919 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,921 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,923 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,925 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,927 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,929 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,931 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,933 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,935 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:28:55,937 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,939 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,941 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,943 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,945 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,947 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,949 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,951 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,953 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,955 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,957 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,959 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,961 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,964 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,966 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,969 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,971 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,973 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,975 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,977 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,979 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,982 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,984 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,987 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,989 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,991 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,993 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,995 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,997 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:55,999 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,002 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,004 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,006 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,008 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,010 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,013 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,015 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,017 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,019 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,022 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,024 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,026 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,028 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,030 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,033 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,034 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,036 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,038 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,040 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,042 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,044 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,046 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,049 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,051 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,053 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,055 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,057 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,059 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,061 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,063 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,065 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,067 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:28:56,069 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,071 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,074 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,076 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,078 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,080 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,082 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,084 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,087 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,088 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,090 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,092 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,094 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,096 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,098 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,100 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,102 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,104 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,106 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,109 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,111 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,113 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,115 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,117 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,119 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,121 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,123 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,126 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,128 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,130 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,132 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,134 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,136 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,138 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,140 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,142 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,144 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,146 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,148 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,150 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,152 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,154 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,157 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,159 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,161 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,163 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,165 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,167 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,169 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,171 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,173 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,175 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,177 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,180 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,182 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,184 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,186 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,188 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,190 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,192 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,194 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,196 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:28:56,198 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,200 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,202 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,204 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,206 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,208 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,210 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,212 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,214 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,216 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,218 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,220 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,222 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,224 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,226 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,228 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,230 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,232 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,234 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,236 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,238 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,240 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,242 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,244 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,247 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,249 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,250 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,252 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,254 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,256 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,258 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,260 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,262 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,264 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,266 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,268 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,270 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,272 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,274 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,276 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,278 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,280 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,282 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,284 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,286 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,288 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,290 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,292 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,295 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,297 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,299 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,301 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,303 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,305 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,307 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,308 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,310 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,312 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,314 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,316 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,318 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,320 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:28:56,322 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,324 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,326 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,328 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,330 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,332 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,334 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,336 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,338 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,341 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,343 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,346 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,348 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,350 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,353 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,355 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,358 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,360 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,362 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,364 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,366 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,369 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,370 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,372 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,374 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,376 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,379 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,381 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,382 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,385 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,387 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,389 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,391 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,393 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,395 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,397 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,399 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,402 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,404 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,406 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,408 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,410 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,412 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,414 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,416 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,418 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,420 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,422 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,424 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,426 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,428 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,430 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,432 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,434 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,436 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,438 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,440 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,443 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,444 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,447 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,449 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,451 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:28:56,453 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,455 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,457 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,459 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,461 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,463 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,465 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,467 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,469 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,471 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,473 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,475 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,477 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,479 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,481 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,483 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,485 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,487 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,489 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,491 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,493 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,495 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,497 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,499 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,502 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,504 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,506 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,508 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,510 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,512 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,514 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,516 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,518 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,520 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,522 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,524 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,526 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,528 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,530 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,532 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,534 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,536 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,538 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,540 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,543 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,545 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,547 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,549 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,551 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,553 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,555 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,557 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,559 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,562 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,564 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,566 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,568 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,570 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,572 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,574 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,576 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,578 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:28:56,580 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,581 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,583 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,585 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,588 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,590 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,592 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,594 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,596 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,598 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,600 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,602 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,604 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,607 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,609 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,611 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,613 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,615 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,617 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,619 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,621 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,623 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,625 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,628 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,629 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,631 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,633 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,636 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,638 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,640 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,642 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,644 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,646 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,648 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,650 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,652 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,654 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,656 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,658 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,660 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,662 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,664 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,666 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,669 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,671 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,673 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,676 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,678 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,680 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,682 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,684 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,686 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,688 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,690 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,692 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,694 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,696 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,698 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,700 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,702 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,704 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,706 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:28:56,708 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,710 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,712 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,715 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,717 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,719 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,722 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,724 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,726 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,728 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,730 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,732 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,734 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,736 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,738 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,740 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,742 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,744 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,746 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,748 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,750 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,752 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,754 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,757 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,759 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,761 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,763 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,765 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,767 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,769 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,771 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,773 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,775 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,777 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,780 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,782 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,784 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,786 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,788 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,790 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,792 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,794 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,797 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,799 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,801 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,803 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,805 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,807 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,809 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,811 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,814 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,815 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,818 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,820 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,822 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,824 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,826 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,829 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,831 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,833 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,835 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,838 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:28:56,840 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,842 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,844 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,846 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,848 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,850 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,852 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,854 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,857 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,859 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,862 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,864 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,866 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,868 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,870 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,872 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,874 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,876 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,878 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,881 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,883 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,885 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,887 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,889 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,892 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,894 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,896 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,898 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,900 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,902 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,905 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,907 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,909 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,911 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,913 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,915 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,917 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,920 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,923 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,925 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,927 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,930 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,933 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,935 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,937 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,940 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,942 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,944 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,946 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,949 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,951 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,953 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,955 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,957 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,959 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,961 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,965 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,967 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,969 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,971 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,973 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,975 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:28:56,977 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,979 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,981 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,983 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,985 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,988 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,990 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,993 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,997 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:56,999 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,001 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,003 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,005 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,007 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,010 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,012 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,014 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,016 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,018 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,020 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,023 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,024 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,026 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,028 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,031 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,033 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,035 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,037 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,039 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,041 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,043 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,045 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,047 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,049 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,051 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,053 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,055 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,057 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,059 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,061 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,063 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,066 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,068 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,071 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,073 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,076 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,078 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,080 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,082 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,084 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,087 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,089 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,091 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,093 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,095 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,097 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,099 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,101 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,104 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,106 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,108 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,111 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:28:57,113 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,115 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,117 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,119 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,122 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,124 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,126 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,128 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,130 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,132 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,134 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,137 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,139 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,141 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,143 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,145 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,148 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,150 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,152 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,154 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,156 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,158 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,161 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,163 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,165 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,167 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,169 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,171 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,173 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,175 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,178 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,180 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,183 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,185 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,187 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,189 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,191 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,193 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,195 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,197 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,199 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,201 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,203 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,205 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,207 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,209 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,212 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,214 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,217 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,220 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,222 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,224 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,226 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,228 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,230 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,233 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,235 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,237 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,239 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,241 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,243 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,245 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:28:57,247 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,249 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,251 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,253 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,255 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,257 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,259 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,261 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,264 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,266 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,268 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,270 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,272 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,274 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,276 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,278 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,280 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,283 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,285 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,287 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,289 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,291 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,293 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,295 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,297 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,299 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,301 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,303 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,305 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,307 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,309 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,311 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,313 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,315 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,317 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,319 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,321 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,323 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,325 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,327 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,330 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,332 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,334 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,336 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,338 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,340 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,342 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,344 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,346 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,348 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,350 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,352 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,354 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,357 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,359 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,361 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,362 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,364 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,367 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,369 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,371 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,373 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:28:57,375 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,377 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,379 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,381 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,384 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,386 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,388 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,390 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,392 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,394 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,396 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,399 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,401 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,403 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,405 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,407 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,409 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,411 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,413 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,415 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,417 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,419 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,421 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,423 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,425 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,427 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,429 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,432 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,434 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,435 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,438 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,440 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,442 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,444 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,446 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,448 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,450 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,452 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,454 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,456 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,458 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,460 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,462 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,464 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,466 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,468 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,470 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,472 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,474 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,476 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,478 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,481 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,483 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,485 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,487 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,489 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,491 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,493 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,495 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,497 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,499 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,502 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:28:57,504 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,507 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,509 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,512 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,514 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,516 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,518 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,520 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,523 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,525 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,528 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,531 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,533 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,535 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,537 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,539 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,541 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,543 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,545 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,547 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,549 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,551 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,553 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,555 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,558 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,560 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,562 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,564 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,567 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,569 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,571 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,573 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,575 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,577 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,579 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,581 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,583 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,585 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,587 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,589 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,592 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,594 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,596 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,598 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,600 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,602 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,604 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,606 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,608 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,611 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,613 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,615 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,617 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,619 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,621 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,623 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,625 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,627 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,629 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,632 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,634 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,636 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:28:57,638 copying lm_eval/tasks/global_mmlu/default/de/_global_mmlu_de.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:57,640 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:57,642 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:57,644 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:57,646 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:57,648 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:57,650 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:28:57,652 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:57,654 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:57,656 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:57,659 copying lm_eval/tasks/global_mmlu/default/hi/_global_mmlu_hi.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:57,661 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:57,663 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:57,665 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:28:57,668 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:57,670 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:57,672 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:57,674 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:57,676 copying lm_eval/tasks/global_mmlu/default/bn/_global_mmlu_bn.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:57,678 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:57,681 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:28:57,683 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:57,685 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:57,687 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:57,689 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:57,691 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:57,693 copying lm_eval/tasks/global_mmlu/default/ar/_global_mmlu_ar.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:57,695 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:28:57,697 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:57,699 copying lm_eval/tasks/global_mmlu/default/es/_global_mmlu_es.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:57,701 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:57,703 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:57,705 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:57,708 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:57,710 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:28:57,712 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:57,714 copying lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:57,716 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:57,718 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:57,720 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:57,722 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:57,724 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:28:57,726 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:57,728 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:57,730 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:57,732 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:57,735 copying lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:57,737 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:57,739 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:28:57,741 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:57,743 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:57,745 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:57,747 copying lm_eval/tasks/global_mmlu/default/id/_global_mmlu_id.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:57,749 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:57,751 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:57,753 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:28:57,755 copying lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:57,758 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:57,760 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:57,761 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:57,764 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:57,766 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:57,768 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:28:57,769 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:57,772 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:57,774 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:57,776 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:57,778 copying lm_eval/tasks/global_mmlu/default/ja/_global_mmlu_ja.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:57,780 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:57,782 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:28:57,784 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:57,787 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:57,789 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:57,791 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:57,793 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:57,795 copying lm_eval/tasks/global_mmlu/default/sw/_global_mmlu_sw.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:57,797 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:28:57,799 copying lm_eval/tasks/global_mmlu/default/fr/_global_mmlu_fr.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:57,801 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:57,803 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:57,805 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:57,807 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:57,809 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:57,811 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:28:57,813 copying lm_eval/tasks/global_mmlu/default/en/_global_mmlu_en.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:57,815 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:57,817 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:57,819 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:57,821 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:57,823 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:57,825 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:28:57,827 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:57,829 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:57,831 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:57,833 copying lm_eval/tasks/global_mmlu/default/yo/_global_mmlu_yo.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:57,836 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:57,838 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:57,840 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:28:57,842 copying lm_eval/tasks/global_mmlu/default/it/_global_mmlu_it.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:57,844 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:57,846 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:57,849 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:57,851 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:57,853 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:57,855 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:28:57,857 copying lm_eval/tasks/mlqa/mlqa_ar_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,859 copying lm_eval/tasks/mlqa/mlqa_vi_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,861 copying lm_eval/tasks/mlqa/mlqa_ar_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,863 copying lm_eval/tasks/mlqa/mlqa_zh_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,865 copying lm_eval/tasks/mlqa/mlqa_en_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,867 copying lm_eval/tasks/mlqa/mlqa_hi_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,869 copying lm_eval/tasks/mlqa/mlqa_es_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,871 copying lm_eval/tasks/mlqa/mlqa_vi_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,873 copying lm_eval/tasks/mlqa/mlqa_vi_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,875 copying lm_eval/tasks/mlqa/mlqa_zh_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,877 copying lm_eval/tasks/mlqa/mlqa_de_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,879 copying lm_eval/tasks/mlqa/mlqa_zh_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,881 copying lm_eval/tasks/mlqa/mlqa_zh_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,883 copying lm_eval/tasks/mlqa/mlqa_en_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,885 copying lm_eval/tasks/mlqa/mlqa_ar_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,887 copying lm_eval/tasks/mlqa/mlqa_de_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,889 copying lm_eval/tasks/mlqa/mlqa_es_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,891 copying lm_eval/tasks/mlqa/mlqa_hi_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,893 copying lm_eval/tasks/mlqa/mlqa_ar_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,896 copying lm_eval/tasks/mlqa/mlqa_de_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,898 copying lm_eval/tasks/mlqa/mlqa_es_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,900 copying lm_eval/tasks/mlqa/mlqa_en_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,902 copying lm_eval/tasks/mlqa/mlqa_hi_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,904 copying lm_eval/tasks/mlqa/mlqa_en_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,906 copying lm_eval/tasks/mlqa/mlqa_vi_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,909 copying lm_eval/tasks/mlqa/mlqa_es_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,911 copying lm_eval/tasks/mlqa/mlqa_hi_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,913 copying lm_eval/tasks/mlqa/mlqa_hi_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,915 copying lm_eval/tasks/mlqa/mlqa_es_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,918 copying lm_eval/tasks/mlqa/mlqa_hi_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,920 copying lm_eval/tasks/mlqa/mlqa_es_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,922 copying lm_eval/tasks/mlqa/mlqa_de_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,924 copying lm_eval/tasks/mlqa/mlqa_de_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,926 copying lm_eval/tasks/mlqa/mlqa_es_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,928 copying lm_eval/tasks/mlqa/mlqa_ar_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,930 copying lm_eval/tasks/mlqa/mlqa_en_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,933 copying lm_eval/tasks/mlqa/mlqa_de_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,935 copying lm_eval/tasks/mlqa/mlqa_en_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,937 copying lm_eval/tasks/mlqa/mlqa_zh_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,939 copying lm_eval/tasks/mlqa/mlqa_en_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,941 copying lm_eval/tasks/mlqa/mlqa_zh_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,943 copying lm_eval/tasks/mlqa/mlqa_vi_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,945 copying lm_eval/tasks/mlqa/mlqa_ar_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,947 copying lm_eval/tasks/mlqa/mlqa_vi_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,949 copying lm_eval/tasks/mlqa/mlqa_hi_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,951 copying lm_eval/tasks/mlqa/mlqa_ar_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,953 copying lm_eval/tasks/mlqa/mlqa_vi_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,955 copying lm_eval/tasks/mlqa/mlqa_zh_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,957 copying lm_eval/tasks/mlqa/mlqa_de_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:28:57,959 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jsquad.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,961 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jaqket_v2.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,963 copying lm_eval/tasks/japanese_leaderboard/_ja_leaderboard.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,965 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,967 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,969 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,971 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,973 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jnli.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,975 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_marc_ja.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:28:57,977 copying lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T14:28:57,979 copying lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T14:28:57,981 copying lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T14:28:57,983 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:28:57,985 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:28:57,988 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:28:57,990 copying lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:28:57,992 copying lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:28:57,994 copying lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:28:57,996 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:28:57,998 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:28:58,000 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:28:58,002 copying lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:28:58,004 copying lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:28:58,006 copying lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:28:58,008 copying lm_eval/tasks/portuguese_bench/assin_entailment.yaml -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T14:28:58,010 copying lm_eval/tasks/portuguese_bench/portuguese_bench.yaml -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T14:28:58,012 copying lm_eval/tasks/portuguese_bench/assin_paraphrase.yaml -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T14:28:58,014 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-eu.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,016 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_gl-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,018 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_fr-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,020 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-de.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,022 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_it-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,024 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_en-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,026 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-es.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,029 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-fr.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,031 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_es-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,033 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_de-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,035 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-en.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,037 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_ca-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,039 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-ca.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,041 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_eu-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,043 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,045 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-gl.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,047 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-it.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:28:58,049 copying lm_eval/tasks/ruler/fwe.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,051 copying lm_eval/tasks/ruler/vt.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,054 copying lm_eval/tasks/ruler/qa_hotpot.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,056 copying lm_eval/tasks/ruler/niah_single_2.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,058 copying lm_eval/tasks/ruler/niah_multikey_1.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,060 copying lm_eval/tasks/ruler/niah_single_1.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,062 copying lm_eval/tasks/ruler/niah_single_3.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,064 copying lm_eval/tasks/ruler/qa_squad.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,067 copying lm_eval/tasks/ruler/cwe.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,069 copying lm_eval/tasks/ruler/ruler.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,071 copying lm_eval/tasks/ruler/niah_multikey_3.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,074 copying lm_eval/tasks/ruler/niah_multiquery.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,076 copying lm_eval/tasks/ruler/niah_multikey_2.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,079 copying lm_eval/tasks/ruler/niah_multivalue.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:28:58,081 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,083 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,086 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,088 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,090 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,092 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,094 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,096 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,098 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,100 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,102 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,104 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,106 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,108 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,110 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,112 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,114 copying lm_eval/tasks/mmlu_prox/de/_mmlu_prox_lite_de.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,116 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,118 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,120 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,122 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,124 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,127 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,129 copying lm_eval/tasks/mmlu_prox/de/_mmlu_prox_de.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,132 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,134 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,136 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,138 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,140 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,142 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:28:58,144 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,146 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,148 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,150 copying lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_lite_hi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,152 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,155 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,157 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,159 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,161 copying lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_hi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,163 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,165 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,167 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,169 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,171 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,173 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,175 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,178 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,180 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,182 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,184 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,186 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,188 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,190 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,193 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,195 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,198 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,238 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,241 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,243 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,245 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:28:58,246 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,248 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,250 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,252 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,254 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,256 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,259 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,261 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,262 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,264 copying lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_lite_ur.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,266 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,268 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,271 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,273 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,275 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,277 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,279 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,281 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,283 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,285 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,287 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,289 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,291 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,293 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,296 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,298 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,300 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,302 copying lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_ur.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,304 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,306 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:28:58,308 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,310 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,312 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,314 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,316 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,318 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,320 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,322 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,325 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,327 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,329 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,331 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,333 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,335 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,337 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,339 copying lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_bn.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,341 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,343 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,345 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,347 copying lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_lite_bn.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,349 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,352 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,354 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,356 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,358 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,360 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,362 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,364 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,366 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,369 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:28:58,371 copying lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_lite_ar.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,373 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,375 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,377 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,379 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,381 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,383 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,385 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,387 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,389 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,391 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,393 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,395 copying lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_ar.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,397 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,399 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,401 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,403 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,405 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,408 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,410 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,412 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,414 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,416 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,418 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,420 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,422 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,424 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,426 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,428 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,431 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:28:58,433 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,435 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,437 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,439 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,441 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,443 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,445 copying lm_eval/tasks/mmlu_prox/es/_mmlu_prox_es.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,447 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,449 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,451 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,453 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,456 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,458 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,460 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,462 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,464 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,466 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,468 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,470 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,472 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,474 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,476 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,478 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,480 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,482 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,484 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,487 copying lm_eval/tasks/mmlu_prox/es/_mmlu_prox_lite_es.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,489 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,491 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,493 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:28:58,495 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,497 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,500 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,502 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,505 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,508 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,511 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,513 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,515 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,517 copying lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_lite_pt.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,519 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,521 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,523 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,525 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,527 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,529 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,532 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,534 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,536 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,538 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,540 copying lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_pt.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,542 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,545 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,547 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,549 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,552 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,554 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,556 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,558 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,560 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:28:58,562 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,565 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,567 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,569 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,571 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,573 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,575 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,577 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,579 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,581 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,583 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,586 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,588 copying lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_zu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,590 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,592 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,594 copying lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_lite_zu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,596 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,599 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,601 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,603 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,605 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,608 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,610 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,612 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,614 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,616 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,618 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,620 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,622 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,625 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:28:58,627 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,629 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,631 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,633 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,636 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,638 copying lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_cs.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,641 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,643 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,645 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,647 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,649 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,651 copying lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_lite_cs.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,653 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,656 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,658 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,661 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,663 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,665 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,668 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,670 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,672 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,674 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,676 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,678 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,680 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,683 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,685 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,688 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,690 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,692 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:28:58,694 copying lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_lite_mr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,696 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,698 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,701 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,703 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,705 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,707 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,709 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,711 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,713 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,716 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,718 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,720 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,722 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,724 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,726 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,728 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,730 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,732 copying lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_mr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,734 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,736 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,739 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,741 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,743 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,745 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,747 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,749 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,751 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,753 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,755 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:28:58,757 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,759 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,761 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,763 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,765 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,768 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,770 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,772 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,774 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,776 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,778 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,780 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,782 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,784 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,786 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,788 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,790 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,792 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,794 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,796 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,798 copying lm_eval/tasks/mmlu_prox/af/_mmlu_prox_af.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,800 copying lm_eval/tasks/mmlu_prox/af/_mmlu_prox_lite_af.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,802 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,804 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,806 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,808 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,810 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,812 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,814 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,816 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:28:58,818 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,820 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,823 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,825 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,827 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,829 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,831 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,833 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,836 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,838 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,840 copying lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_lite_wo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,842 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,844 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,846 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,848 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,850 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,853 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,855 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,857 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,859 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,861 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,863 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,866 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,868 copying lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_wo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,870 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,872 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,874 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,876 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,878 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,880 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:28:58,882 copying lm_eval/tasks/mmlu_prox/th/_mmlu_prox_th.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,884 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,886 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,888 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,890 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,892 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,894 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,896 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,898 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,901 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,903 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,905 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,907 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,909 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,911 copying lm_eval/tasks/mmlu_prox/th/_mmlu_prox_lite_th.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,913 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,915 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,917 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,919 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,921 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,923 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,925 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,928 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,930 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,932 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,934 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,936 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,938 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,940 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,943 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:28:58,945 copying lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_ru.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,947 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,949 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,951 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,954 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,956 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,958 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,960 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,962 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,964 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,966 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,968 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,970 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,972 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,975 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,976 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,978 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,980 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,982 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,984 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,986 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,988 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,990 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,992 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,994 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,996 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:58,998 copying lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_lite_ru.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:59,000 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:59,002 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:59,004 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:28:59,006 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,008 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,010 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,012 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,014 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,016 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,018 copying lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_lite_sr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,020 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,022 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,024 copying lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_sr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,026 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,028 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,031 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,033 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,035 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,037 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,039 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,041 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,043 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,045 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,047 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,049 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,051 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,053 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,055 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,057 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,059 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,061 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,063 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,065 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:28:59,067 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,069 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,071 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,073 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,075 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,077 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,080 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,082 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,084 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,086 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,088 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,090 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,092 copying lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_lite_zh.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,094 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,096 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,098 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,100 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,102 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,104 copying lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_zh.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,106 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,108 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,110 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,112 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,114 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,115 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,117 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,119 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,121 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,123 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,126 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:28:59,128 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,130 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,132 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,134 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,136 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,138 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,141 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,143 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,145 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,147 copying lm_eval/tasks/mmlu_prox/id/_mmlu_prox_id.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,149 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,151 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,153 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,155 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,157 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,159 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,161 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,164 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,166 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,168 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,170 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,173 copying lm_eval/tasks/mmlu_prox/id/_mmlu_prox_lite_id.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,175 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,177 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,179 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,181 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,183 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,185 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,187 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,188 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:28:59,190 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,192 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,194 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,196 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,198 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,200 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,203 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,204 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,207 copying lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_lite_ko.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,209 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,211 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,214 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,216 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,218 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,221 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,223 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,225 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,228 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,230 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,232 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,235 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,238 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,240 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,242 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,245 copying lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_ko.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,247 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,250 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,251 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,253 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,256 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:28:59,258 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,260 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,262 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,264 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,266 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,268 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,270 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,272 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,274 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,276 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,278 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,280 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,283 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,285 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,287 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,289 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,291 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,293 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,296 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,298 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,300 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,302 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,304 copying lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_ja.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,307 copying lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_lite_ja.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,309 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,311 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,313 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,315 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,317 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,319 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:28:59,321 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,323 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,325 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,327 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,329 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,331 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,333 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,335 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,337 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,340 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,342 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,344 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,347 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,349 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,351 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,353 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,355 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,358 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,360 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,362 copying lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_lite_uk.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,364 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,366 copying lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_uk.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,369 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,371 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,373 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,375 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,377 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,379 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,381 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,384 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:28:59,385 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,388 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,390 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,392 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,394 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,396 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,398 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,400 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,403 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,405 copying lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_lite_sw.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,407 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,409 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,411 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,413 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,415 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,418 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,420 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,422 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,424 copying lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_sw.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,426 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,428 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,431 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,433 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,435 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,437 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,439 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,441 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,443 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,446 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,448 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:28:59,450 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,452 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,454 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,456 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,458 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,460 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,462 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,464 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,467 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,469 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,471 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,473 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,475 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,477 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,479 copying lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_lite_fr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,481 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,483 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,485 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,487 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,489 copying lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_fr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,491 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,493 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,495 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,497 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,499 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,501 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,503 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,505 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,507 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,509 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:28:59,511 copying lm_eval/tasks/mmlu_prox/en/_mmlu_prox_lite_en.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,513 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,515 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,517 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,519 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,521 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,523 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,525 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,527 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,529 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,531 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,533 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,535 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,538 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,540 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,542 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,545 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,547 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,549 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,551 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,553 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,555 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,557 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,559 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,562 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,564 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,566 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,568 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,570 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,572 copying lm_eval/tasks/mmlu_prox/en/_mmlu_prox_en.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:28:59,574 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,576 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,578 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,580 copying lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_lite_ne.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,582 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,584 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,586 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,588 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,590 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,592 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,594 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,596 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,598 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,600 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,602 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,604 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,606 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,608 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,610 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,612 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,614 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,616 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,618 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,620 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,622 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,625 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,627 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,629 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,632 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,634 copying lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_ne.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:28:59,636 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,638 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,640 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,642 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,644 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,646 copying lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_lite_yo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,648 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,651 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,653 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,655 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,657 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,660 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,662 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,664 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,666 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,668 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,670 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,672 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,674 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,676 copying lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_yo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,678 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,680 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,682 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,684 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,686 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,688 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,690 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,693 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,695 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,697 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:28:59,700 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,702 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,704 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,706 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,708 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,710 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,712 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,714 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,716 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,718 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,720 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,722 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,724 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,726 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,729 copying lm_eval/tasks/mmlu_prox/te/_mmlu_prox_te.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,731 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,733 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,735 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,738 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,740 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,742 copying lm_eval/tasks/mmlu_prox/te/_mmlu_prox_lite_te.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,744 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,746 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,749 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,751 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,753 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,755 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,757 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,759 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,762 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:28:59,764 copying lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_vi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,766 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,769 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,772 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,774 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,776 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,778 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,781 copying lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_lite_vi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,782 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,784 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,787 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,789 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,791 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,793 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,795 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,797 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,799 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,801 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,803 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,805 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,808 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,810 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,812 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,815 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,817 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,820 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,822 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,825 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,828 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,830 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:28:59,832 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,834 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,837 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,839 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,841 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,844 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,845 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,847 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,849 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,851 copying lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_hu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,853 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,856 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,858 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,860 copying lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_lite_hu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,862 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,864 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,866 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,868 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,870 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,872 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,874 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,876 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,879 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,881 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,883 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,885 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,887 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,889 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,891 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,893 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:28:59,895 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,897 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,899 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,901 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,904 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,906 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,908 copying lm_eval/tasks/mmlu_prox/it/_mmlu_prox_it.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,910 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,912 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,914 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,917 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,919 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,921 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,923 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,925 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,927 copying lm_eval/tasks/mmlu_prox/it/_mmlu_prox_lite_it.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,929 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,932 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,934 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,936 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,938 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,940 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,942 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,944 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,946 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,949 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,951 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,953 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,955 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,957 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:28:59,960 copying lm_eval/tasks/swde/swde.yaml -> build/lib/lm_eval/tasks/swde 2026-05-11T14:28:59,963 creating build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,177 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,179 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,182 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,185 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,187 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,189 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,191 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,193 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,195 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,197 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,199 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,201 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,204 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,206 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,209 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,212 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,214 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,217 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,219 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,221 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,223 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,225 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,227 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,229 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,231 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,233 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,235 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,237 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,239 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,241 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,243 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,245 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,247 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,249 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,252 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,254 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,256 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_other.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,258 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,260 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,262 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,264 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,266 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,268 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,270 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,272 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,273 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,275 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,280 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,284 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,286 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:00,289 creating build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,290 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,507 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,511 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,514 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,518 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,521 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,525 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,528 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,532 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,535 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,538 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,542 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,545 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,549 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,552 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,556 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,560 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,563 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,566 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,570 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,573 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,577 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,580 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_other.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,584 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,587 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,590 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,592 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,594 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,596 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,598 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,600 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,602 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,604 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,606 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,608 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,610 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,613 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,615 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,617 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,619 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,621 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,623 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,625 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,627 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,629 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,631 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,633 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,635 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,637 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,639 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:00,641 creating build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,642 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,645 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,647 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,650 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,653 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,655 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,658 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,660 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,663 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,665 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,668 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,671 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,673 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,676 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,679 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,681 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,683 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,686 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,689 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,692 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,694 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,697 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,700 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,702 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,705 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,708 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,710 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,713 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,715 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,718 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,721 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,723 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,726 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,728 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,731 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,733 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,736 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,739 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,741 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,743 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,746 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,749 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,751 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,754 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,756 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,759 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,762 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,764 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,766 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_other.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,769 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:00,771 creating build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,772 copying lm_eval/tasks/kmmlu/default/kmmlu_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,774 copying lm_eval/tasks/kmmlu/default/kmmlu_math.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,776 copying lm_eval/tasks/kmmlu/default/kmmlu_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,779 copying lm_eval/tasks/kmmlu/default/kmmlu_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,780 copying lm_eval/tasks/kmmlu/default/kmmlu_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,783 copying lm_eval/tasks/kmmlu/default/kmmlu_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,785 copying lm_eval/tasks/kmmlu/default/kmmlu_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,787 copying lm_eval/tasks/kmmlu/default/kmmlu_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,789 copying lm_eval/tasks/kmmlu/default/kmmlu_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,791 copying lm_eval/tasks/kmmlu/default/kmmlu_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,793 copying lm_eval/tasks/kmmlu/default/kmmlu_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,795 copying lm_eval/tasks/kmmlu/default/kmmlu_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,797 copying lm_eval/tasks/kmmlu/default/_kmmlu_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,807 copying lm_eval/tasks/kmmlu/default/kmmlu_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,809 copying lm_eval/tasks/kmmlu/default/kmmlu_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,811 copying lm_eval/tasks/kmmlu/default/kmmlu_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,813 copying lm_eval/tasks/kmmlu/default/kmmlu_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,815 copying lm_eval/tasks/kmmlu/default/kmmlu_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,817 copying lm_eval/tasks/kmmlu/default/kmmlu_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,819 copying lm_eval/tasks/kmmlu/default/kmmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,821 copying lm_eval/tasks/kmmlu/default/kmmlu_management.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,823 copying lm_eval/tasks/kmmlu/default/kmmlu_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,825 copying lm_eval/tasks/kmmlu/default/kmmlu_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,827 copying lm_eval/tasks/kmmlu/default/kmmlu_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,829 copying lm_eval/tasks/kmmlu/default/kmmlu_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,831 copying lm_eval/tasks/kmmlu/default/kmmlu_education.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,833 copying lm_eval/tasks/kmmlu/default/kmmlu_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,835 copying lm_eval/tasks/kmmlu/default/kmmlu_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,838 copying lm_eval/tasks/kmmlu/default/_kmmlu_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,840 copying lm_eval/tasks/kmmlu/default/kmmlu_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,842 copying lm_eval/tasks/kmmlu/default/kmmlu_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,844 copying lm_eval/tasks/kmmlu/default/_kmmlu_other.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,846 copying lm_eval/tasks/kmmlu/default/kmmlu_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,848 copying lm_eval/tasks/kmmlu/default/kmmlu_health.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,850 copying lm_eval/tasks/kmmlu/default/kmmlu_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,852 copying lm_eval/tasks/kmmlu/default/_kmmlu_default.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,854 copying lm_eval/tasks/kmmlu/default/kmmlu_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,856 copying lm_eval/tasks/kmmlu/default/kmmlu_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,858 copying lm_eval/tasks/kmmlu/default/kmmlu_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,860 copying lm_eval/tasks/kmmlu/default/kmmlu_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,862 copying lm_eval/tasks/kmmlu/default/kmmlu_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,863 copying lm_eval/tasks/kmmlu/default/kmmlu_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,865 copying lm_eval/tasks/kmmlu/default/kmmlu_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,867 copying lm_eval/tasks/kmmlu/default/_kmmlu_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,869 copying lm_eval/tasks/kmmlu/default/kmmlu_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,871 copying lm_eval/tasks/kmmlu/default/kmmlu_law.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,873 copying lm_eval/tasks/kmmlu/default/kmmlu_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,875 copying lm_eval/tasks/kmmlu/default/kmmlu_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,877 copying lm_eval/tasks/kmmlu/default/kmmlu_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,879 copying lm_eval/tasks/kmmlu/default/kmmlu_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:00,881 creating build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,882 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,884 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,887 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,889 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,891 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,893 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,895 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,897 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,899 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,901 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,903 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,905 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,907 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,909 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,911 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,913 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,915 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,917 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,919 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,921 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,923 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,925 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,927 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,929 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,931 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,933 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,935 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,937 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,939 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,941 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,943 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,945 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,947 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,949 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,951 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_other.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,953 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,955 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,957 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,959 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,961 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,963 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,965 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,967 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,969 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,971 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,973 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,975 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,977 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,979 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,981 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:00,983 copying lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:00,985 copying lm_eval/tasks/arabicmmlu/arabicmmlu_biology_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:00,988 copying lm_eval/tasks/arabicmmlu/arabicmmlu_math_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:00,990 copying lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:00,992 copying lm_eval/tasks/arabicmmlu/arabicmmlu_political_science_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:00,994 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:00,996 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:00,998 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,000 copying lm_eval/tasks/arabicmmlu/arabicmmlu_history_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,002 copying lm_eval/tasks/arabicmmlu/arabicmmlu_management_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,004 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,006 copying lm_eval/tasks/arabicmmlu/arabicmmlu_geography_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,008 copying lm_eval/tasks/arabicmmlu/arabicmmlu_geography_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,010 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_humanities.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,012 copying lm_eval/tasks/arabicmmlu/arabicmmlu_accounting_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,015 copying lm_eval/tasks/arabicmmlu/arabicmmlu_economics_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,017 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_other.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,019 copying lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,022 copying lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,024 copying lm_eval/tasks/arabicmmlu/arabicmmlu_civics_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,026 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_social_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,029 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,031 copying lm_eval/tasks/arabicmmlu/arabicmmlu_history_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,033 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_stem.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,035 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,037 copying lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,040 copying lm_eval/tasks/arabicmmlu/arabicmmlu_physics_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,042 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,044 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,047 copying lm_eval/tasks/arabicmmlu/_arabicmmlu.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,049 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,050 copying lm_eval/tasks/arabicmmlu/arabicmmlu_economics_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,053 copying lm_eval/tasks/arabicmmlu/arabicmmlu_civics_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,055 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,057 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,059 copying lm_eval/tasks/arabicmmlu/arabicmmlu_philosophy_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,061 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_language.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,063 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,065 copying lm_eval/tasks/arabicmmlu/arabicmmlu_geography_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,067 copying lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,069 copying lm_eval/tasks/arabicmmlu/arabicmmlu_history_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,071 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,073 copying lm_eval/tasks/arabicmmlu/arabicmmlu_economics_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,075 copying lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,077 copying lm_eval/tasks/arabicmmlu/arabicmmlu_law_professional.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,079 copying lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:01,081 copying lm_eval/tasks/noticia/noticia.yaml -> build/lib/lm_eval/tasks/noticia 2026-05-11T14:29:01,083 creating build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,084 copying lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,087 copying lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,089 copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,091 copying lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,093 copying lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,095 copying lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,097 copying lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,099 copying lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,101 copying lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,103 copying lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,105 copying lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,107 copying lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,108 copying lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,110 copying lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,112 copying lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,114 copying lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,116 copying lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,118 copying lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,120 copying lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,122 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,124 copying lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,127 copying lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,129 copying lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,131 copying lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,133 copying lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,135 copying lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,137 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,139 copying lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,141 copying lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,143 copying lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,145 copying lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,147 copying lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,149 copying lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,151 copying lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,153 copying lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,155 copying lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,157 copying lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,159 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,161 copying lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,163 copying lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,165 copying lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,167 copying lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,169 copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,171 copying lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,173 copying lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,175 copying lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,177 copying lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,179 copying lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,181 copying lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,183 copying lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,185 copying lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,187 copying lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,189 copying lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,191 copying lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,193 copying lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,195 copying lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,198 copying lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,200 copying lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,202 copying lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,204 copying lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,206 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,208 copying lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,210 copying lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,212 copying lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,213 copying lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,215 copying lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,217 copying lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,219 copying lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,221 copying lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,223 copying lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,225 copying lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,227 copying lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,230 copying lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,232 copying lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,234 copying lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,236 copying lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,238 copying lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,240 copying lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,242 copying lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,244 copying lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,246 copying lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,248 copying lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,250 copying lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,252 copying lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,254 copying lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,257 copying lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,259 copying lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,261 copying lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,263 copying lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,265 copying lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,267 copying lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,269 copying lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,271 copying lm_eval/tasks/bigbench/generate_until/color.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,273 copying lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,274 copying lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,276 copying lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,278 copying lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,281 copying lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,283 copying lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,285 copying lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,287 copying lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,289 copying lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,291 copying lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,293 copying lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,295 copying lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,297 copying lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,299 copying lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,301 copying lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,303 copying lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,305 copying lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,308 copying lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,310 copying lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,312 copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,314 copying lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,316 copying lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,318 copying lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,320 copying lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,322 copying lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,324 copying lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,326 copying lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,328 copying lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,329 copying lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,331 copying lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,333 copying lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,336 copying lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,338 copying lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,340 copying lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,342 copying lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,344 copying lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,346 copying lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,348 copying lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,351 copying lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,353 copying lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,355 copying lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,357 copying lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,359 copying lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,361 copying lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,363 copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,365 copying lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,367 copying lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,369 copying lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,371 copying lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,374 copying lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,376 copying lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,378 copying lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,380 copying lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,381 copying lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,383 copying lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,385 copying lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,387 copying lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,389 copying lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,391 copying lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,393 copying lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,395 copying lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,397 copying lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,400 copying lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,402 copying lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,404 copying lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,406 copying lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,408 copying lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,410 copying lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,412 copying lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,414 copying lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,416 copying lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,418 copying lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,421 copying lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,423 copying lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:01,425 creating build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,426 copying lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,428 copying lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,430 copying lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,432 copying lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,434 copying lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,437 copying lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,439 copying lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,441 copying lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,443 copying lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,445 copying lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,447 copying lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,449 copying lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,451 copying lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,453 copying lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,455 copying lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,457 copying lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,459 copying lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,461 copying lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,463 copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,465 copying lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,468 copying lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,470 copying lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,472 copying lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,474 copying lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,476 copying lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,479 copying lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,481 copying lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,483 copying lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,485 copying lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,487 copying lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,489 copying lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,491 copying lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,493 copying lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,495 copying lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,498 copying lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,500 copying lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,502 copying lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,504 copying lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,506 copying lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,508 copying lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,511 copying lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,517 copying lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,519 copying lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,521 copying lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,523 copying lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,525 copying lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,527 copying lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,529 copying lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,531 copying lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,534 copying lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,536 copying lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,538 copying lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,540 copying lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,543 copying lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,545 copying lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,548 copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,550 copying lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,552 copying lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,554 copying lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,556 copying lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,558 copying lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,561 copying lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,563 copying lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,565 copying lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,567 copying lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,569 copying lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,571 copying lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,573 copying lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,575 copying lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,577 copying lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,579 copying lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,581 copying lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,583 copying lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,585 copying lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,588 copying lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,590 copying lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,592 copying lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,594 copying lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,596 copying lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,599 copying lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,601 copying lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,604 copying lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,606 copying lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,609 copying lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,611 copying lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,613 copying lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,615 copying lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,618 copying lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,620 copying lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,623 copying lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,625 copying lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,627 copying lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,629 copying lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,631 copying lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,633 copying lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,635 copying lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,638 copying lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,641 copying lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,644 copying lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,646 copying lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,648 copying lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,650 copying lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,652 copying lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,654 copying lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,656 copying lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,658 copying lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,661 copying lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,663 copying lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,665 copying lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,667 copying lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,670 copying lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,672 copying lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,674 copying lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,676 copying lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,678 copying lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,680 copying lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,682 copying lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,684 copying lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,686 copying lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:01,688 copying lm_eval/tasks/global_piqa/completions/fin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,690 copying lm_eval/tasks/global_piqa/completions/azj_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,692 copying lm_eval/tasks/global_piqa/completions/spa_latn_mexi.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,694 copying lm_eval/tasks/global_piqa/completions/slk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,696 copying lm_eval/tasks/global_piqa/completions/deu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,698 copying lm_eval/tasks/global_piqa/completions/nob_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,700 copying lm_eval/tasks/global_piqa/completions/swe_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,702 copying lm_eval/tasks/global_piqa/completions/jpn_jpan.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,704 copying lm_eval/tasks/global_piqa/completions/lin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,707 copying lm_eval/tasks/global_piqa/completions/glg_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,709 copying lm_eval/tasks/global_piqa/completions/apc_arab_syri.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,711 copying lm_eval/tasks/global_piqa/completions/ars_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,713 copying lm_eval/tasks/global_piqa/completions/slv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,715 copying lm_eval/tasks/global_piqa/completions/por_latn_port.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,717 copying lm_eval/tasks/global_piqa/completions/bsk_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,719 copying lm_eval/tasks/global_piqa/completions/tur_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,721 copying lm_eval/tasks/global_piqa/completions/zsm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,723 copying lm_eval/tasks/global_piqa/completions/ekk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,725 copying lm_eval/tasks/global_piqa/completions/tgl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,727 copying lm_eval/tasks/global_piqa/completions/pes_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,729 copying lm_eval/tasks/global_piqa/completions/tam_taml.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,732 copying lm_eval/tasks/global_piqa/completions/hau_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,734 copying lm_eval/tasks/global_piqa/completions/kan_knda.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,736 copying lm_eval/tasks/global_piqa/completions/pan_guru.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,738 copying lm_eval/tasks/global_piqa/completions/ben_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,739 copying lm_eval/tasks/global_piqa/completions/swh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,741 copying lm_eval/tasks/global_piqa/completions/sin_sinh.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,743 copying lm_eval/tasks/global_piqa/completions/mkd_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,745 copying lm_eval/tasks/global_piqa/completions/bel_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,747 copying lm_eval/tasks/global_piqa/completions/mni_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,749 copying lm_eval/tasks/global_piqa/completions/cmn_hans.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,752 copying lm_eval/tasks/global_piqa/completions/mni_mtei.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,753 copying lm_eval/tasks/global_piqa/completions/hye_armn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,756 copying lm_eval/tasks/global_piqa/completions/yor_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,758 copying lm_eval/tasks/global_piqa/completions/fra_latn_fran.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,760 copying lm_eval/tasks/global_piqa/completions/ben_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,762 copying lm_eval/tasks/global_piqa/completions/slv_latn_cerk.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,764 copying lm_eval/tasks/global_piqa/completions/als_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,766 copying lm_eval/tasks/global_piqa/completions/arb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,768 copying lm_eval/tasks/global_piqa/completions/hun_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,770 copying lm_eval/tasks/global_piqa/completions/aeb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,772 copying lm_eval/tasks/global_piqa/completions/ibo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,775 copying lm_eval/tasks/global_piqa/completions/kat_geor.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,777 copying lm_eval/tasks/global_piqa/completions/idu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,779 copying lm_eval/tasks/global_piqa/completions/guj_gujr.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,781 copying lm_eval/tasks/global_piqa/completions/cmn_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,783 copying lm_eval/tasks/global_piqa/completions/jav_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,785 copying lm_eval/tasks/global_piqa/completions/mal_mlym.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,787 copying lm_eval/tasks/global_piqa/completions/ces_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,789 copying lm_eval/tasks/global_piqa/completions/tel_telu.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,791 copying lm_eval/tasks/global_piqa/completions/fra_latn_cana.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,794 copying lm_eval/tasks/global_piqa/completions/arq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,796 copying lm_eval/tasks/global_piqa/completions/afb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,798 copying lm_eval/tasks/global_piqa/completions/kor_hang.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,799 copying lm_eval/tasks/global_piqa/completions/ary_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,801 copying lm_eval/tasks/global_piqa/completions/ckm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,803 copying lm_eval/tasks/global_piqa/completions/haw_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,805 copying lm_eval/tasks/global_piqa/completions/hrv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,807 copying lm_eval/tasks/global_piqa/completions/bul_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,809 copying lm_eval/tasks/global_piqa/completions/ekp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,811 copying lm_eval/tasks/global_piqa/completions/iso_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,813 copying lm_eval/tasks/global_piqa/completions/slk_latn_sari.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,815 copying lm_eval/tasks/global_piqa/completions/asm_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,817 copying lm_eval/tasks/global_piqa/completions/lit_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,819 copying lm_eval/tasks/global_piqa/completions/ukr_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,821 copying lm_eval/tasks/global_piqa/completions/nag_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,823 copying lm_eval/tasks/global_piqa/completions/yue_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,826 copying lm_eval/tasks/global_piqa/completions/dhd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,828 copying lm_eval/tasks/global_piqa/completions/kir_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,830 copying lm_eval/tasks/global_piqa/completions/vie_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,832 copying lm_eval/tasks/global_piqa/completions/amh_ethi.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,834 copying lm_eval/tasks/global_piqa/completions/eng_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,836 copying lm_eval/tasks/global_piqa/completions/luo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,838 copying lm_eval/tasks/global_piqa/completions/apc_arab_jord.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,841 copying lm_eval/tasks/global_piqa/completions/bho_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,844 copying lm_eval/tasks/global_piqa/completions/bam_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,847 copying lm_eval/tasks/global_piqa/completions/pol_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,849 copying lm_eval/tasks/global_piqa/completions/arz_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,851 copying lm_eval/tasks/global_piqa/completions/srp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,853 copying lm_eval/tasks/global_piqa/completions/urd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,855 copying lm_eval/tasks/global_piqa/completions/cat_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,858 copying lm_eval/tasks/global_piqa/completions/ckb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,860 copying lm_eval/tasks/global_piqa/completions/ita_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,862 copying lm_eval/tasks/global_piqa/completions/srp_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,864 copying lm_eval/tasks/global_piqa/completions/pcm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,867 copying lm_eval/tasks/global_piqa/completions/ind_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,869 copying lm_eval/tasks/global_piqa/completions/npi_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,871 copying lm_eval/tasks/global_piqa/completions/urd_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,874 copying lm_eval/tasks/global_piqa/completions/heb_hebr.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,876 copying lm_eval/tasks/global_piqa/completions/rwr_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,878 copying lm_eval/tasks/global_piqa/completions/apc_arab_leba.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,880 copying lm_eval/tasks/global_piqa/completions/_global_piqa.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,883 copying lm_eval/tasks/global_piqa/completions/kaz_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,885 copying lm_eval/tasks/global_piqa/completions/spa_latn_spai.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,888 copying lm_eval/tasks/global_piqa/completions/ron_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,890 copying lm_eval/tasks/global_piqa/completions/mar_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,893 copying lm_eval/tasks/global_piqa/completions/isl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,895 copying lm_eval/tasks/global_piqa/completions/nld_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,897 copying lm_eval/tasks/global_piqa/completions/snd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,899 copying lm_eval/tasks/global_piqa/completions/ell_grek.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,901 copying lm_eval/tasks/global_piqa/completions/tha_thai.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,903 copying lm_eval/tasks/global_piqa/completions/urh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,905 copying lm_eval/tasks/global_piqa/completions/hin_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,907 copying lm_eval/tasks/global_piqa/completions/por_latn_braz.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,909 copying lm_eval/tasks/global_piqa/completions/uzn_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,911 copying lm_eval/tasks/global_piqa/completions/acq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,913 copying lm_eval/tasks/global_piqa/completions/kin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,915 copying lm_eval/tasks/global_piqa/completions/apc_arab_pale.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,917 copying lm_eval/tasks/global_piqa/completions/uig_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,919 copying lm_eval/tasks/global_piqa/completions/spa_latn_peru.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,921 copying lm_eval/tasks/global_piqa/completions/bos_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,923 copying lm_eval/tasks/global_piqa/completions/fao_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,925 copying lm_eval/tasks/global_piqa/completions/acm_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,927 copying lm_eval/tasks/global_piqa/completions/zul_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,929 copying lm_eval/tasks/global_piqa/completions/rus_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,931 copying lm_eval/tasks/global_piqa/completions/nno_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,933 copying lm_eval/tasks/global_piqa/completions/snd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:01,935 copying lm_eval/tasks/global_piqa/prompted/fin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,937 copying lm_eval/tasks/global_piqa/prompted/azj_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,939 copying lm_eval/tasks/global_piqa/prompted/spa_latn_mexi.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,941 copying lm_eval/tasks/global_piqa/prompted/slk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,944 copying lm_eval/tasks/global_piqa/prompted/deu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,945 copying lm_eval/tasks/global_piqa/prompted/nob_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,947 copying lm_eval/tasks/global_piqa/prompted/swe_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,949 copying lm_eval/tasks/global_piqa/prompted/jpn_jpan.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,951 copying lm_eval/tasks/global_piqa/prompted/lin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,953 copying lm_eval/tasks/global_piqa/prompted/glg_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,955 copying lm_eval/tasks/global_piqa/prompted/apc_arab_syri.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,957 copying lm_eval/tasks/global_piqa/prompted/ars_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,960 copying lm_eval/tasks/global_piqa/prompted/slv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,962 copying lm_eval/tasks/global_piqa/prompted/por_latn_port.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,963 copying lm_eval/tasks/global_piqa/prompted/bsk_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,966 copying lm_eval/tasks/global_piqa/prompted/tur_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,968 copying lm_eval/tasks/global_piqa/prompted/zsm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,970 copying lm_eval/tasks/global_piqa/prompted/ekk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,972 copying lm_eval/tasks/global_piqa/prompted/tgl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,974 copying lm_eval/tasks/global_piqa/prompted/pes_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,976 copying lm_eval/tasks/global_piqa/prompted/tam_taml.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,978 copying lm_eval/tasks/global_piqa/prompted/hau_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,981 copying lm_eval/tasks/global_piqa/prompted/kan_knda.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,984 copying lm_eval/tasks/global_piqa/prompted/pan_guru.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,986 copying lm_eval/tasks/global_piqa/prompted/ben_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,989 copying lm_eval/tasks/global_piqa/prompted/swh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,991 copying lm_eval/tasks/global_piqa/prompted/sin_sinh.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,993 copying lm_eval/tasks/global_piqa/prompted/mkd_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,995 copying lm_eval/tasks/global_piqa/prompted/bel_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,997 copying lm_eval/tasks/global_piqa/prompted/mni_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:01,999 copying lm_eval/tasks/global_piqa/prompted/cmn_hans.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,001 copying lm_eval/tasks/global_piqa/prompted/mni_mtei.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,003 copying lm_eval/tasks/global_piqa/prompted/hye_armn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,005 copying lm_eval/tasks/global_piqa/prompted/yor_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,007 copying lm_eval/tasks/global_piqa/prompted/fra_latn_fran.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,008 copying lm_eval/tasks/global_piqa/prompted/ben_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,010 copying lm_eval/tasks/global_piqa/prompted/slv_latn_cerk.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,012 copying lm_eval/tasks/global_piqa/prompted/als_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,014 copying lm_eval/tasks/global_piqa/prompted/arb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,016 copying lm_eval/tasks/global_piqa/prompted/hun_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,018 copying lm_eval/tasks/global_piqa/prompted/aeb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,021 copying lm_eval/tasks/global_piqa/prompted/ibo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,023 copying lm_eval/tasks/global_piqa/prompted/kat_geor.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,027 copying lm_eval/tasks/global_piqa/prompted/idu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,029 copying lm_eval/tasks/global_piqa/prompted/guj_gujr.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,031 copying lm_eval/tasks/global_piqa/prompted/cmn_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,033 copying lm_eval/tasks/global_piqa/prompted/jav_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,035 copying lm_eval/tasks/global_piqa/prompted/mal_mlym.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,037 copying lm_eval/tasks/global_piqa/prompted/ces_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,039 copying lm_eval/tasks/global_piqa/prompted/tel_telu.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,041 copying lm_eval/tasks/global_piqa/prompted/fra_latn_cana.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,043 copying lm_eval/tasks/global_piqa/prompted/arq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,045 copying lm_eval/tasks/global_piqa/prompted/afb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,047 copying lm_eval/tasks/global_piqa/prompted/kor_hang.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,049 copying lm_eval/tasks/global_piqa/prompted/ary_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,051 copying lm_eval/tasks/global_piqa/prompted/ckm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,053 copying lm_eval/tasks/global_piqa/prompted/haw_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,055 copying lm_eval/tasks/global_piqa/prompted/hrv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,057 copying lm_eval/tasks/global_piqa/prompted/bul_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,059 copying lm_eval/tasks/global_piqa/prompted/ekp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,061 copying lm_eval/tasks/global_piqa/prompted/iso_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,063 copying lm_eval/tasks/global_piqa/prompted/slk_latn_sari.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,065 copying lm_eval/tasks/global_piqa/prompted/asm_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,067 copying lm_eval/tasks/global_piqa/prompted/lit_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,069 copying lm_eval/tasks/global_piqa/prompted/ukr_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,071 copying lm_eval/tasks/global_piqa/prompted/nag_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,073 copying lm_eval/tasks/global_piqa/prompted/yue_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,075 copying lm_eval/tasks/global_piqa/prompted/dhd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,077 copying lm_eval/tasks/global_piqa/prompted/kir_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,079 copying lm_eval/tasks/global_piqa/prompted/vie_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,081 copying lm_eval/tasks/global_piqa/prompted/amh_ethi.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,083 copying lm_eval/tasks/global_piqa/prompted/eng_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,085 copying lm_eval/tasks/global_piqa/prompted/luo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,087 copying lm_eval/tasks/global_piqa/prompted/apc_arab_jord.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,089 copying lm_eval/tasks/global_piqa/prompted/bho_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,092 copying lm_eval/tasks/global_piqa/prompted/bam_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,094 copying lm_eval/tasks/global_piqa/prompted/pol_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,096 copying lm_eval/tasks/global_piqa/prompted/arz_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,098 copying lm_eval/tasks/global_piqa/prompted/srp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,100 copying lm_eval/tasks/global_piqa/prompted/urd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,102 copying lm_eval/tasks/global_piqa/prompted/cat_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,104 copying lm_eval/tasks/global_piqa/prompted/ckb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,106 copying lm_eval/tasks/global_piqa/prompted/ita_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,108 copying lm_eval/tasks/global_piqa/prompted/srp_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,110 copying lm_eval/tasks/global_piqa/prompted/pcm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,112 copying lm_eval/tasks/global_piqa/prompted/ind_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,114 copying lm_eval/tasks/global_piqa/prompted/npi_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,116 copying lm_eval/tasks/global_piqa/prompted/urd_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,118 copying lm_eval/tasks/global_piqa/prompted/heb_hebr.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,120 copying lm_eval/tasks/global_piqa/prompted/rwr_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,122 copying lm_eval/tasks/global_piqa/prompted/apc_arab_leba.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,124 copying lm_eval/tasks/global_piqa/prompted/_global_piqa.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,126 copying lm_eval/tasks/global_piqa/prompted/kaz_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,128 copying lm_eval/tasks/global_piqa/prompted/spa_latn_spai.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,130 copying lm_eval/tasks/global_piqa/prompted/ron_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,132 copying lm_eval/tasks/global_piqa/prompted/mar_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,134 copying lm_eval/tasks/global_piqa/prompted/isl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,136 copying lm_eval/tasks/global_piqa/prompted/nld_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,138 copying lm_eval/tasks/global_piqa/prompted/snd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,140 copying lm_eval/tasks/global_piqa/prompted/ell_grek.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,142 copying lm_eval/tasks/global_piqa/prompted/tha_thai.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,144 copying lm_eval/tasks/global_piqa/prompted/urh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,146 copying lm_eval/tasks/global_piqa/prompted/hin_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,148 copying lm_eval/tasks/global_piqa/prompted/por_latn_braz.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,151 copying lm_eval/tasks/global_piqa/prompted/uzn_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,153 copying lm_eval/tasks/global_piqa/prompted/acq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,155 copying lm_eval/tasks/global_piqa/prompted/kin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,157 copying lm_eval/tasks/global_piqa/prompted/apc_arab_pale.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,159 copying lm_eval/tasks/global_piqa/prompted/uig_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,161 copying lm_eval/tasks/global_piqa/prompted/spa_latn_peru.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,163 copying lm_eval/tasks/global_piqa/prompted/bos_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,165 copying lm_eval/tasks/global_piqa/prompted/fao_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,167 copying lm_eval/tasks/global_piqa/prompted/acm_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,169 copying lm_eval/tasks/global_piqa/prompted/zul_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,171 copying lm_eval/tasks/global_piqa/prompted/rus_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,173 copying lm_eval/tasks/global_piqa/prompted/nno_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,175 copying lm_eval/tasks/global_piqa/prompted/snd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:02,177 creating build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,178 copying lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,181 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,183 copying lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,186 copying lm_eval/tasks/bbh/cot_fewshot/_bbh_cot_fewshot.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,188 copying lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,191 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,193 copying lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,195 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,198 copying lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,200 copying lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,202 copying lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,205 copying lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,207 copying lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,209 copying lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,211 copying lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,213 copying lm_eval/tasks/bbh/cot_fewshot/_bbh.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,215 copying lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,217 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,220 copying lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,222 copying lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,224 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,226 copying lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,228 copying lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,231 copying lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,233 copying lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,235 copying lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,237 copying lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,240 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,242 copying lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:02,244 creating build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,245 copying lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,247 copying lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,250 copying lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,252 copying lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,254 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,256 copying lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,258 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,260 copying lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,262 copying lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,264 copying lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,266 copying lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,268 copying lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,270 copying lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,272 copying lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,274 copying lm_eval/tasks/bbh/fewshot/_bbh_fewshot.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,277 copying lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,279 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,281 copying lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,283 copying lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,285 copying lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,287 copying lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,289 copying lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,291 copying lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,293 copying lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,295 copying lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,298 copying lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,300 copying lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,302 copying lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:02,305 copying lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,307 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,309 copying lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,311 copying lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,313 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,315 copying lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,317 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,319 copying lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,321 copying lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,323 copying lm_eval/tasks/bbh/cot_zeroshot/_bbh_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,325 copying lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,327 copying lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,329 copying lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,331 copying lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,333 copying lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,335 copying lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,337 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,339 copying lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,341 copying lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,343 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,345 copying lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,347 copying lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,349 copying lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,351 copying lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,353 copying lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,355 copying lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,358 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,360 copying lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:02,362 copying lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,364 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,366 copying lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,368 copying lm_eval/tasks/bbh/zeroshot/_bbh_zeroshot.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,370 copying lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,372 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,374 copying lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,377 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,379 copying lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,381 copying lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,383 copying lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,385 copying lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,386 copying lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,388 copying lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,390 copying lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,392 copying lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,394 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,396 copying lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,398 copying lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,400 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,402 copying lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,404 copying lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,406 copying lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,408 copying lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,410 copying lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,412 copying lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,414 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,416 copying lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:02,418 creating build/lib/lm_eval/tasks/babi 2026-05-11T14:29:02,419 copying lm_eval/tasks/babi/babi.yaml -> build/lib/lm_eval/tasks/babi 2026-05-11T14:29:02,422 copying lm_eval/tasks/drop/default.yaml -> build/lib/lm_eval/tasks/drop 2026-05-11T14:29:02,424 copying lm_eval/tasks/toxigen/toxigen.yaml -> build/lib/lm_eval/tasks/toxigen 2026-05-11T14:29:02,426 copying lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/lib/lm_eval/tasks/pubmedqa 2026-05-11T14:29:02,428 creating build/lib/lm_eval/tasks/anli 2026-05-11T14:29:02,428 copying lm_eval/tasks/anli/anli_r3.yaml -> build/lib/lm_eval/tasks/anli 2026-05-11T14:29:02,431 copying lm_eval/tasks/anli/anli_r2.yaml -> build/lib/lm_eval/tasks/anli 2026-05-11T14:29:02,433 copying lm_eval/tasks/anli/anli_r1.yaml -> build/lib/lm_eval/tasks/anli 2026-05-11T14:29:02,435 copying lm_eval/tasks/infinitebench/infinitebench_math_find.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,437 copying lm_eval/tasks/infinitebench/infinitebench_code_debug.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,439 copying lm_eval/tasks/infinitebench/infinitebench_longbook_qa_chn.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,441 copying lm_eval/tasks/infinitebench/infinitebench.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,443 copying lm_eval/tasks/infinitebench/infinitebench_longbook_choice_en.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,445 copying lm_eval/tasks/infinitebench/infinitebench_number_string.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,447 copying lm_eval/tasks/infinitebench/infinitebench_passkey.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,449 copying lm_eval/tasks/infinitebench/infinitebench_code_run.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,451 copying lm_eval/tasks/infinitebench/infinitebench_longbook_qa_en.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,453 copying lm_eval/tasks/infinitebench/infinitebench_longdialogue_qa_en.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,455 copying lm_eval/tasks/infinitebench/infinitebench_longbook_sum_en.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,457 copying lm_eval/tasks/infinitebench/infinitebench_kv_retrieval.yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:02,459 copying lm_eval/tasks/agieval/agieval_cn.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,461 copying lm_eval/tasks/agieval/lsat-rc.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,463 copying lm_eval/tasks/agieval/math.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,465 copying lm_eval/tasks/agieval/aqua-rat.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,467 copying lm_eval/tasks/agieval/jec-qa-kd.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,469 copying lm_eval/tasks/agieval/lsat-ar.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,471 copying lm_eval/tasks/agieval/agieval_nous.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,473 copying lm_eval/tasks/agieval/agieval_en.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,475 copying lm_eval/tasks/agieval/logiqa-en.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,477 copying lm_eval/tasks/agieval/jec-qa-ca.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,479 copying lm_eval/tasks/agieval/gaokao-chinese.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,481 copying lm_eval/tasks/agieval/gaokao-english.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,483 copying lm_eval/tasks/agieval/sat-en.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,485 copying lm_eval/tasks/agieval/lsat-lr.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,487 copying lm_eval/tasks/agieval/gaokao-chemistry.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,488 copying lm_eval/tasks/agieval/gaokao-mathqa.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,491 copying lm_eval/tasks/agieval/gaokao-biology.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,492 copying lm_eval/tasks/agieval/gaokao-history.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,494 copying lm_eval/tasks/agieval/gaokao-mathcloze.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,496 copying lm_eval/tasks/agieval/gaokao-physics.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,498 copying lm_eval/tasks/agieval/logiqa-zh.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,500 copying lm_eval/tasks/agieval/sat-en-without-passage.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,502 copying lm_eval/tasks/agieval/agieval.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,505 copying lm_eval/tasks/agieval/sat-math.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,507 copying lm_eval/tasks/agieval/gaokao-geography.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:02,509 creating build/lib/lm_eval/tasks/prost 2026-05-11T14:29:02,510 copying lm_eval/tasks/prost/corypaik_prost.yaml -> build/lib/lm_eval/tasks/prost 2026-05-11T14:29:02,512 copying lm_eval/tasks/unitxt/argument_topic.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,514 copying lm_eval/tasks/unitxt/ag_news.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,516 copying lm_eval/tasks/unitxt/unfair_tos.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,518 copying lm_eval/tasks/unitxt/medical_abstracts.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,520 copying lm_eval/tasks/unitxt/dbpedia_14.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,522 copying lm_eval/tasks/unitxt/ledgar.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,524 copying lm_eval/tasks/unitxt/financial_tweets.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,526 copying lm_eval/tasks/unitxt/cnn_dailymail.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,528 copying lm_eval/tasks/unitxt/xsum.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,530 copying lm_eval/tasks/unitxt/yahoo_answers_topics.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,532 copying lm_eval/tasks/unitxt/law_stack_exchange.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,534 copying lm_eval/tasks/unitxt/banking77.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,536 copying lm_eval/tasks/unitxt/20_newsgroups.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,538 copying lm_eval/tasks/unitxt/claim_stance_topic.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,540 copying lm_eval/tasks/unitxt/atis.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,542 copying lm_eval/tasks/unitxt/stsb.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,544 copying lm_eval/tasks/unitxt/ethos_binary.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,546 copying lm_eval/tasks/unitxt/coedit_gec.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,548 copying lm_eval/tasks/unitxt/doc_vqa.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:02,550 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,552 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,554 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,556 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,558 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,560 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,562 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,564 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,566 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,568 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,570 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,572 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,574 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,577 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,579 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,581 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,583 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,585 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,587 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,589 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,591 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,593 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,595 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,597 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,599 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,601 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,603 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,605 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,607 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,609 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,611 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,613 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,615 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,618 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,620 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,622 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,624 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,626 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,628 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,630 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,632 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,634 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,636 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,638 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,640 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,642 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,644 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,646 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,648 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:02,650 creating build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:02,651 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:02,653 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:02,655 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:02,657 copying lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,659 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,661 copying lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,663 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,665 copying lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,668 copying lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,670 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,672 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,674 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,676 copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,678 copying lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,680 copying lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,682 copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,684 copying lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,685 copying lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,688 copying lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,690 copying lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,692 copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,693 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,695 copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,697 copying lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,699 copying lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,701 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,703 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,706 copying lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,708 copying lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,710 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,712 copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,714 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,716 copying lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,718 copying lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,720 copying lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,722 copying lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,724 copying lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,726 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,728 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,730 copying lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,732 copying lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,734 copying lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,736 copying lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,738 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,740 copying lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,742 copying lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,744 copying lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,746 copying lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,748 copying lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,750 copying lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,752 copying lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,754 copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,757 copying lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,759 copying lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,761 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,764 copying lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,766 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,768 copying lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,770 copying lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,772 copying lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,774 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,777 copying lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,779 copying lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,781 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,784 copying lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,786 copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,788 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,790 copying lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,792 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,794 copying lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,796 copying lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,798 copying lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,800 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,802 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,804 copying lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,806 copying lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,808 copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,810 copying lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,812 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,814 copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,817 copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,819 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,821 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,823 copying lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,825 copying lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,827 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,829 copying lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,831 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,833 copying lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,835 copying lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,837 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,839 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,841 copying lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,843 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,845 copying lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,847 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,849 copying lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,851 copying lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,853 copying lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,855 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,857 copying lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,859 copying lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,861 copying lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,863 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,865 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,867 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,869 copying lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,871 copying lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,873 copying lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,875 copying lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,877 copying lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,879 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,881 copying lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,883 copying lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,885 copying lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,887 copying lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,889 copying lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,891 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,893 copying lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,895 copying lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,897 copying lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,899 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,901 copying lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,903 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,906 copying lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,908 copying lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,910 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,912 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,914 copying lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,916 copying lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,918 copying lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,920 copying lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,922 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,924 copying lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,926 copying lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,928 copying lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,930 copying lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,931 copying lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:02,933 copying lm_eval/tasks/metabench/metabench_arc_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,935 copying lm_eval/tasks/metabench/metabench_arc_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,937 copying lm_eval/tasks/metabench/metabench_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,939 copying lm_eval/tasks/metabench/metabench_gsm8k.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,941 copying lm_eval/tasks/metabench/metabench_winogrande_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,944 copying lm_eval/tasks/metabench/metabench_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,946 copying lm_eval/tasks/metabench/metabench_winogrande_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,950 copying lm_eval/tasks/metabench/metabench_arc_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,953 copying lm_eval/tasks/metabench/metabench_truthfulqa_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,955 copying lm_eval/tasks/metabench/metabench_winogrande.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,957 copying lm_eval/tasks/metabench/metabench_mmlu_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,959 copying lm_eval/tasks/metabench/metabench_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,961 copying lm_eval/tasks/metabench/metabench_mmlu.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,963 copying lm_eval/tasks/metabench/metabench_hellaswag_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,965 copying lm_eval/tasks/metabench/metabench_truthfulqa_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,967 copying lm_eval/tasks/metabench/metabench_gsm8k_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,969 copying lm_eval/tasks/metabench/metabench_hellaswag_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,972 copying lm_eval/tasks/metabench/metabench_winogrande_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,974 copying lm_eval/tasks/metabench/metabench_hellaswag_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,976 copying lm_eval/tasks/metabench/metabench_mmlu_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,978 copying lm_eval/tasks/metabench/metabench.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,980 copying lm_eval/tasks/metabench/metabench_mmlu_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,982 copying lm_eval/tasks/metabench/metabench_arc.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,984 copying lm_eval/tasks/metabench/metabench_truthfulqa_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,986 copying lm_eval/tasks/metabench/metabench_hellaswag.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,988 copying lm_eval/tasks/metabench/metabench_truthfulqa.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:02,990 copying lm_eval/tasks/cmmlu/cmmlu_agronomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:02,993 copying lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:02,995 copying lm_eval/tasks/cmmlu/cmmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:02,997 copying lm_eval/tasks/cmmlu/cmmlu_chinese_foreign_policy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:02,999 copying lm_eval/tasks/cmmlu/cmmlu_chinese_literature.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,001 copying lm_eval/tasks/cmmlu/cmmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,003 copying lm_eval/tasks/cmmlu/cmmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,005 copying lm_eval/tasks/cmmlu/cmmlu_education.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,007 copying lm_eval/tasks/cmmlu/cmmlu_world_history.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,009 copying lm_eval/tasks/cmmlu/cmmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,011 copying lm_eval/tasks/cmmlu/cmmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,013 copying lm_eval/tasks/cmmlu/cmmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,015 copying lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,017 copying lm_eval/tasks/cmmlu/cmmlu_virology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,019 copying lm_eval/tasks/cmmlu/cmmlu_legal_and_moral_basis.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,021 copying lm_eval/tasks/cmmlu/cmmlu_chinese_teacher_qualification.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,023 copying lm_eval/tasks/cmmlu/cmmlu_ethnology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,025 copying lm_eval/tasks/cmmlu/cmmlu_ancient_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,027 copying lm_eval/tasks/cmmlu/cmmlu_college_engineering_hydrology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,029 copying lm_eval/tasks/cmmlu/cmmlu_sociology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,031 copying lm_eval/tasks/cmmlu/cmmlu_logical.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,033 copying lm_eval/tasks/cmmlu/cmmlu_international_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,035 copying lm_eval/tasks/cmmlu/cmmlu_security_study.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,037 copying lm_eval/tasks/cmmlu/cmmlu_economics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,039 copying lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,041 copying lm_eval/tasks/cmmlu/cmmlu_food_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,043 copying lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,045 copying lm_eval/tasks/cmmlu/cmmlu_computer_security.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,047 copying lm_eval/tasks/cmmlu/cmmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,049 copying lm_eval/tasks/cmmlu/cmmlu_chinese_history.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,051 copying lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,053 copying lm_eval/tasks/cmmlu/cmmlu_elementary_commonsense.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,055 copying lm_eval/tasks/cmmlu/cmmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,057 copying lm_eval/tasks/cmmlu/cmmlu_astronomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,059 copying lm_eval/tasks/cmmlu/cmmlu_arts.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,061 copying lm_eval/tasks/cmmlu/cmmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,063 copying lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,065 copying lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,067 copying lm_eval/tasks/cmmlu/cmmlu_elementary_information_and_technology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,069 copying lm_eval/tasks/cmmlu/cmmlu_marketing.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,071 copying lm_eval/tasks/cmmlu/cmmlu_management.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,073 copying lm_eval/tasks/cmmlu/cmmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,075 copying lm_eval/tasks/cmmlu/cmmlu_journalism.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,077 copying lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,079 copying lm_eval/tasks/cmmlu/cmmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,081 copying lm_eval/tasks/cmmlu/cmmlu_chinese_food_culture.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,083 copying lm_eval/tasks/cmmlu/cmmlu_college_actuarial_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,085 copying lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,087 copying lm_eval/tasks/cmmlu/cmmlu_college_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,089 copying lm_eval/tasks/cmmlu/cmmlu_chinese_civil_service_exam.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,091 copying lm_eval/tasks/cmmlu/cmmlu_high_school_politics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,093 copying lm_eval/tasks/cmmlu/cmmlu_anatomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,095 copying lm_eval/tasks/cmmlu/cmmlu_elementary_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,097 copying lm_eval/tasks/cmmlu/cmmlu_college_medical_statistics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,100 copying lm_eval/tasks/cmmlu/cmmlu_college_education.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,102 copying lm_eval/tasks/cmmlu/cmmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,104 copying lm_eval/tasks/cmmlu/_cmmlu.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,106 copying lm_eval/tasks/cmmlu/cmmlu_chinese_driving_rule.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,108 copying lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,110 copying lm_eval/tasks/cmmlu/cmmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,112 copying lm_eval/tasks/cmmlu/cmmlu_global_facts.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,114 copying lm_eval/tasks/cmmlu/cmmlu_genetics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,116 copying lm_eval/tasks/cmmlu/cmmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,118 copying lm_eval/tasks/cmmlu/cmmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,120 copying lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,122 copying lm_eval/tasks/cmmlu/cmmlu_construction_project_management.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,124 copying lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,126 copying lm_eval/tasks/cmmlu/cmmlu_computer_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:03,128 copying lm_eval/tasks/medmcqa/medmcqa.yaml -> build/lib/lm_eval/tasks/medmcqa 2026-05-11T14:29:03,130 copying lm_eval/tasks/paws-x/paws_zh.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,132 copying lm_eval/tasks/paws-x/_pawsx.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,134 copying lm_eval/tasks/paws-x/paws_fr.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,136 copying lm_eval/tasks/paws-x/paws_es.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,138 copying lm_eval/tasks/paws-x/paws_de.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,140 copying lm_eval/tasks/paws-x/paws_ja.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,142 copying lm_eval/tasks/paws-x/paws_en.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,144 copying lm_eval/tasks/paws-x/paws_ko.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:03,146 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:03,148 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:03,150 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:03,152 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:03,154 copying lm_eval/tasks/include/few_shot_en/Indonesian/_include_base_44_indonesian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:03,157 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:03,159 copying lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:03,161 copying lm_eval/tasks/include/few_shot_en/Tagalog/_include_base_44_tagalog.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:03,163 copying lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:03,165 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:03,167 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:03,170 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:03,172 copying lm_eval/tasks/include/few_shot_en/North Macedonian/_include_base_44_north macedonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:03,175 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:03,178 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:03,181 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:03,183 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:03,185 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:03,187 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:03,189 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:03,191 copying lm_eval/tasks/include/few_shot_en/Malayalam/_include_base_44_malayalam.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:03,192 copying lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:03,194 copying lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:03,196 copying lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:03,198 copying lm_eval/tasks/include/few_shot_en/Belarusian/_include_base_44_belarusian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:03,200 copying lm_eval/tasks/include/few_shot_en/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:03,202 copying lm_eval/tasks/include/few_shot_en/Kazakh/_include_base_44_kazakh.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:03,204 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:03,207 copying lm_eval/tasks/include/few_shot_en/Persian/_include_base_44_persian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:03,209 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:03,211 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:03,427 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:03,430 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:03,433 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:03,436 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:03,438 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:03,441 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:03,444 copying lm_eval/tasks/include/few_shot_en/Arabic/_include_base_44_arabic.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:03,447 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:03,449 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:03,452 copying lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:03,455 copying lm_eval/tasks/include/few_shot_en/Ukrainian/_include_base_44_ukrainian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:03,458 copying lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:03,461 copying lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:03,463 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:03,466 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:03,469 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:03,471 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:03,474 copying lm_eval/tasks/include/few_shot_en/Telugu/_include_base_44_telugu.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:03,477 copying lm_eval/tasks/include/few_shot_en/Urdu/_include_base_44_urdu.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:03,480 copying lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:03,482 copying lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:03,485 copying lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:03,488 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:03,491 copying lm_eval/tasks/include/few_shot_en/Lithuanian/_include_base_44_lithuanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:03,493 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:03,496 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:03,499 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:03,502 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:03,504 copying lm_eval/tasks/include/few_shot_en/Vietnamese/_include_base_44_vietnamese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:03,507 copying lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:03,510 copying lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:03,513 copying lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:03,516 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:03,519 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:03,522 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:03,524 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:03,527 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:03,529 copying lm_eval/tasks/include/few_shot_en/Albanian/_include_base_44_albanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:03,531 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:03,533 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:03,535 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:03,537 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:03,540 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:03,543 copying lm_eval/tasks/include/few_shot_en/Finnish/_include_base_44_finnish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:03,546 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:03,549 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:03,551 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:03,553 copying lm_eval/tasks/include/few_shot_en/Spanish/_include_base_44_spanish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:03,555 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:03,557 copying lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:03,559 copying lm_eval/tasks/include/few_shot_en/Nepali/_include_base_44_nepali.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:03,561 copying lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:03,563 copying lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:03,565 copying lm_eval/tasks/include/few_shot_en/Polish/_include_base_44_polish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:03,567 copying lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:03,570 copying lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:03,572 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:03,574 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:03,576 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:03,578 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:03,580 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:03,582 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:03,584 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:03,586 copying lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:03,589 copying lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:03,591 copying lm_eval/tasks/include/few_shot_en/Serbian/_include_base_44_serbian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:03,593 copying lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:03,595 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:03,597 copying lm_eval/tasks/include/few_shot_en/Portuguese/_include_base_44_portuguese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:03,599 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:03,601 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:03,603 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:03,605 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:03,607 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:03,609 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:03,611 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:03,613 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:03,615 copying lm_eval/tasks/include/few_shot_en/French/_include_base_44_french.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:03,617 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:03,619 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:03,621 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:03,623 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:03,625 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:03,627 copying lm_eval/tasks/include/few_shot_en/Uzbek/_include_base_44_uzbek.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:03,629 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:03,631 copying lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:03,633 copying lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:03,635 copying lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:03,637 copying lm_eval/tasks/include/few_shot_en/Malay/_include_base_44_malay.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:03,639 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:03,641 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:03,643 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:03,645 copying lm_eval/tasks/include/few_shot_en/Italian/_include_base_44_italian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:03,647 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:03,649 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:03,651 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:03,653 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:03,655 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:03,658 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:03,660 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:03,662 copying lm_eval/tasks/include/few_shot_en/Estonian/_include_base_44_estonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:03,664 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:03,667 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:03,669 copying lm_eval/tasks/include/few_shot_en/Bengali/_include_base_44_bengali.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:03,671 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:03,673 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:03,675 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:03,677 copying lm_eval/tasks/include/few_shot_en/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:03,679 copying lm_eval/tasks/include/few_shot_en/Georgian/_include_base_44_georgian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:03,681 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,683 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,684 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,687 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,689 copying lm_eval/tasks/include/few_shot_en/Russian/_include_base_44_russian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,691 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,693 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,695 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,697 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:03,699 copying lm_eval/tasks/include/few_shot_en/Korean/_include_base_44_korean.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:03,701 copying lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:03,703 copying lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:03,706 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,708 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,710 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,712 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,714 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,717 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,719 copying lm_eval/tasks/include/few_shot_en/Chinese/_include_base_44_chinese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,721 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,723 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:03,725 copying lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:03,727 copying lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:03,729 copying lm_eval/tasks/include/few_shot_en/Croatian/_include_base_44_croatian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:03,731 copying lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:03,733 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:03,735 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:03,737 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:03,739 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:03,742 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:03,744 copying lm_eval/tasks/include/few_shot_en/Dutch/_include_base_44_dutch.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:03,746 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,748 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,750 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,752 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,754 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,756 copying lm_eval/tasks/include/few_shot_en/Hindi/_include_base_44_hindi.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,758 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,760 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,762 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:03,764 copying lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:03,767 copying lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:03,769 copying lm_eval/tasks/include/few_shot_en/Tamil/_include_base_44_tamil.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:03,771 copying lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:03,773 copying lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:03,775 copying lm_eval/tasks/include/few_shot_en/Hungarian/_include_base_44_hungarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:03,777 copying lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:03,779 copying lm_eval/tasks/include/few_shot_en/Turkish/_include_base_44_turkish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:03,781 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:03,783 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:03,785 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:03,787 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:03,789 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,791 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,793 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,795 copying lm_eval/tasks/include/few_shot_en/Greek/_include_base_44_greek.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,797 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,800 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,802 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,804 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:03,806 copying lm_eval/tasks/include/few_shot_en/Armenian/_include_base_44_armenian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:03,808 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:03,810 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:03,812 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:03,814 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:03,816 copying lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:03,818 copying lm_eval/tasks/include/few_shot_en/Hebrew/_include_base_44_hebrew.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:03,820 copying lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:03,822 copying lm_eval/tasks/include/few_shot_en/German/include_base_44_german_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:03,825 copying lm_eval/tasks/include/few_shot_en/German/include_base_44_german_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:03,827 copying lm_eval/tasks/include/few_shot_en/German/include_base_44_german_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:03,829 copying lm_eval/tasks/include/few_shot_en/German/_include_base_44_german.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:03,831 copying lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:03,833 copying lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:03,835 copying lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:03,837 copying lm_eval/tasks/include/few_shot_en/Japanese/_include_base_44_japanese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:03,839 copying lm_eval/tasks/include/few_shot_en/Basque/include_base_44_basque_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:03,841 copying lm_eval/tasks/include/few_shot_en/Basque/_include_base_44_basque.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:03,843 copying lm_eval/tasks/include/few_shot_en/Bulgarian/_include_base_44_bulgarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:03,845 copying lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:03,847 copying lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:03,849 copying lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:03,851 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:03,853 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:03,855 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:03,857 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:03,860 copying lm_eval/tasks/include/default/Indonesian/_include_base_44_indonesian.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:03,862 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:03,864 copying lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:03,866 copying lm_eval/tasks/include/default/Tagalog/_include_base_44_tagalog.yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:03,868 copying lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:03,870 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:03,872 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:03,874 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:03,876 copying lm_eval/tasks/include/default/North Macedonian/_include_base_44_north macedonian.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:03,878 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:03,880 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:03,882 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:03,884 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_stem.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:03,886 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:03,888 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:03,890 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:03,892 copying lm_eval/tasks/include/default/Malayalam/_include_base_44_malayalam.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:03,894 copying lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:03,896 copying lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:03,898 copying lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:03,900 copying lm_eval/tasks/include/default/Belarusian/_include_base_44_belarusian.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:03,902 copying lm_eval/tasks/include/default/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:03,904 copying lm_eval/tasks/include/default/Kazakh/_include_base_44_kazakh.yaml -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:03,907 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:03,909 copying lm_eval/tasks/include/default/Persian/_include_base_44_persian.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:03,911 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:03,913 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:03,915 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:03,917 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:03,920 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:03,921 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:03,923 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:03,925 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:03,928 copying lm_eval/tasks/include/default/Arabic/_include_base_44_arabic.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:03,930 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:03,933 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_stem.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:03,935 copying lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:03,937 copying lm_eval/tasks/include/default/Ukrainian/_include_base_44_ukrainian.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:03,939 copying lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:03,941 copying lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:03,943 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_stem.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:03,945 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:03,947 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:03,949 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:03,951 copying lm_eval/tasks/include/default/Telugu/_include_base_44_telugu.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:03,953 copying lm_eval/tasks/include/default/Urdu/_include_base_44_urdu.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:03,955 copying lm_eval/tasks/include/default/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:03,957 copying lm_eval/tasks/include/default/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:03,960 copying lm_eval/tasks/include/default/Urdu/include_base_44_urdu_stem.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:03,962 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:03,964 copying lm_eval/tasks/include/default/Lithuanian/_include_base_44_lithuanian.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:03,967 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:03,969 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:03,971 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:03,973 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:03,975 copying lm_eval/tasks/include/default/Vietnamese/_include_base_44_vietnamese.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:03,976 copying lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:03,979 copying lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:03,981 copying lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:03,983 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:03,984 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:03,987 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:03,989 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:03,991 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:03,993 copying lm_eval/tasks/include/default/Albanian/_include_base_44_albanian.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:03,995 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:03,997 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:03,999 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:04,001 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:04,003 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:04,005 copying lm_eval/tasks/include/default/Finnish/_include_base_44_finnish.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:04,007 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:04,009 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:04,011 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:04,013 copying lm_eval/tasks/include/default/Spanish/_include_base_44_spanish.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:04,015 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:04,017 copying lm_eval/tasks/include/default/Nepali/include_base_44_nepali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:04,020 copying lm_eval/tasks/include/default/Nepali/_include_base_44_nepali.yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:04,022 copying lm_eval/tasks/include/default/Nepali/include_base_44_nepali_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:04,024 copying lm_eval/tasks/include/default/Polish/include_base_44_polish_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T14:29:04,026 copying lm_eval/tasks/include/default/Polish/_include_base_44_polish.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T14:29:04,028 copying lm_eval/tasks/include/default/Polish/include_base_44_polish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T14:29:04,029 copying lm_eval/tasks/include/default/Polish/include_base_44_polish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T14:29:04,031 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:04,033 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:04,035 copying lm_eval/tasks/include/default/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:04,037 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:04,040 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:04,042 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:04,044 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:04,046 copying lm_eval/tasks/include/default/Serbian/include_base_44_serbian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:04,048 copying lm_eval/tasks/include/default/Serbian/include_base_44_serbian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:04,050 copying lm_eval/tasks/include/default/Serbian/_include_base_44_serbian.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:04,052 copying lm_eval/tasks/include/default/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:04,054 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:04,056 copying lm_eval/tasks/include/default/Portuguese/_include_base_44_portuguese.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:04,058 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:04,060 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_stem.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:04,062 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:04,064 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:04,067 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:04,069 copying lm_eval/tasks/include/default/French/include_base_44_french_stem.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:29:04,070 copying lm_eval/tasks/include/default/French/include_base_44_french_social_science.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:29:04,072 copying lm_eval/tasks/include/default/French/include_base_44_french_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:29:04,074 copying lm_eval/tasks/include/default/French/_include_base_44_french.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:29:04,077 copying lm_eval/tasks/include/default/French/include_base_44_french_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:29:04,078 copying lm_eval/tasks/include/default/French/include_base_44_french_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:29:04,080 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:04,082 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_stem.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:04,084 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:04,086 copying lm_eval/tasks/include/default/Uzbek/_include_base_44_uzbek.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:04,088 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:04,090 copying lm_eval/tasks/include/default/Malay/include_base_44_malay_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T14:29:04,092 copying lm_eval/tasks/include/default/Malay/include_base_44_malay_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T14:29:04,094 copying lm_eval/tasks/include/default/Malay/include_base_44_malay_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T14:29:04,096 copying lm_eval/tasks/include/default/Malay/_include_base_44_malay.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T14:29:04,098 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:04,100 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:04,102 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:04,104 copying lm_eval/tasks/include/default/Italian/_include_base_44_italian.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:04,106 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:04,108 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:04,110 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:04,112 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:04,114 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:04,117 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:04,119 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:04,121 copying lm_eval/tasks/include/default/Estonian/_include_base_44_estonian.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:04,123 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:04,125 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:04,126 copying lm_eval/tasks/include/default/Bengali/_include_base_44_bengali.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:04,128 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_stem.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:04,130 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:04,132 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:04,134 copying lm_eval/tasks/include/default/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:04,137 copying lm_eval/tasks/include/default/Georgian/_include_base_44_georgian.yaml -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:04,139 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,141 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,143 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,145 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,147 copying lm_eval/tasks/include/default/Russian/_include_base_44_russian.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,150 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,152 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_marine_license.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,154 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,156 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:04,158 copying lm_eval/tasks/include/default/Korean/_include_base_44_korean.yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T14:29:04,161 copying lm_eval/tasks/include/default/Korean/include_base_44_korean_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T14:29:04,163 copying lm_eval/tasks/include/default/Korean/include_base_44_korean_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T14:29:04,165 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,167 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,169 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,171 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,174 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,176 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,178 copying lm_eval/tasks/include/default/Chinese/_include_base_44_chinese.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,180 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_stem.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,182 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:04,185 copying lm_eval/tasks/include/default/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:04,187 copying lm_eval/tasks/include/default/Croatian/include_base_44_croatian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:04,189 copying lm_eval/tasks/include/default/Croatian/_include_base_44_croatian.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:04,191 copying lm_eval/tasks/include/default/Croatian/include_base_44_croatian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:04,193 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:04,196 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_stem.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:04,198 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:04,200 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:04,202 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:04,204 copying lm_eval/tasks/include/default/Dutch/_include_base_44_dutch.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:04,206 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,208 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_stem.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,210 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,212 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,214 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,217 copying lm_eval/tasks/include/default/Hindi/_include_base_44_hindi.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,219 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,221 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,223 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:04,225 copying lm_eval/tasks/include/default/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:04,228 copying lm_eval/tasks/include/default/Tamil/include_base_44_tamil_stem.yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:04,230 copying lm_eval/tasks/include/default/Tamil/_include_base_44_tamil.yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:04,232 copying lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:04,234 copying lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:04,236 copying lm_eval/tasks/include/default/Hungarian/_include_base_44_hungarian.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:04,239 copying lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:04,241 copying lm_eval/tasks/include/default/Turkish/_include_base_44_turkish.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:04,243 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:04,245 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:04,248 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:04,250 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:04,252 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,254 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_stem.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,256 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,258 copying lm_eval/tasks/include/default/Greek/_include_base_44_greek.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,260 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_medical_license.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,262 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,265 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,267 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:04,269 copying lm_eval/tasks/include/default/Armenian/_include_base_44_armenian.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:04,271 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:04,273 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:04,275 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:04,277 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:04,280 copying lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:04,281 copying lm_eval/tasks/include/default/Hebrew/_include_base_44_hebrew.yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:04,283 copying lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:04,285 copying lm_eval/tasks/include/default/German/include_base_44_german_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T14:29:04,287 copying lm_eval/tasks/include/default/German/include_base_44_german_social_science.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T14:29:04,289 copying lm_eval/tasks/include/default/German/include_base_44_german_stem.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T14:29:04,291 copying lm_eval/tasks/include/default/German/_include_base_44_german.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T14:29:04,293 copying lm_eval/tasks/include/default/Japanese/include_base_44_japanese_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:04,295 copying lm_eval/tasks/include/default/Japanese/include_base_44_japanese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:04,298 copying lm_eval/tasks/include/default/Japanese/include_base_44_japanese_medical_license.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:04,300 copying lm_eval/tasks/include/default/Japanese/_include_base_44_japanese.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:04,302 copying lm_eval/tasks/include/default/Basque/include_base_44_basque_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T14:29:04,304 copying lm_eval/tasks/include/default/Basque/_include_base_44_basque.yaml -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T14:29:04,306 copying lm_eval/tasks/include/default/Bulgarian/_include_base_44_bulgarian.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:04,308 copying lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:04,310 copying lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:04,312 copying lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:04,314 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:04,317 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:04,319 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:04,321 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:04,323 copying lm_eval/tasks/include/few_shot_og/Indonesian/_include_base_44_indonesian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:04,325 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:04,327 copying lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:04,329 copying lm_eval/tasks/include/few_shot_og/Tagalog/_include_base_44_tagalog.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:04,331 copying lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:04,333 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:04,335 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:04,337 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:04,339 copying lm_eval/tasks/include/few_shot_og/North Macedonian/_include_base_44_north macedonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:04,341 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:04,343 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:04,345 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:04,347 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:04,349 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:04,351 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:04,354 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:04,356 copying lm_eval/tasks/include/few_shot_og/Malayalam/_include_base_44_malayalam.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:04,358 copying lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:04,360 copying lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:04,362 copying lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:04,364 copying lm_eval/tasks/include/few_shot_og/Belarusian/_include_base_44_belarusian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:04,366 copying lm_eval/tasks/include/few_shot_og/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:04,368 copying lm_eval/tasks/include/few_shot_og/Kazakh/_include_base_44_kazakh.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:04,370 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:04,372 copying lm_eval/tasks/include/few_shot_og/Persian/_include_base_44_persian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:04,374 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:04,376 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:04,378 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:04,380 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:04,383 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:04,385 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:04,387 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:04,389 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:04,391 copying lm_eval/tasks/include/few_shot_og/Arabic/_include_base_44_arabic.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:04,393 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:04,395 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:04,397 copying lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:04,400 copying lm_eval/tasks/include/few_shot_og/Ukrainian/_include_base_44_ukrainian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:04,401 copying lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:04,403 copying lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:04,405 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:04,408 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:04,410 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:04,412 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:04,414 copying lm_eval/tasks/include/few_shot_og/Telugu/_include_base_44_telugu.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:04,416 copying lm_eval/tasks/include/few_shot_og/Urdu/_include_base_44_urdu.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:04,418 copying lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:04,420 copying lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:04,422 copying lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:04,424 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:04,427 copying lm_eval/tasks/include/few_shot_og/Lithuanian/_include_base_44_lithuanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:04,428 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:04,430 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:04,432 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:04,434 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:04,436 copying lm_eval/tasks/include/few_shot_og/Vietnamese/_include_base_44_vietnamese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:04,438 copying lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:04,440 copying lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:04,442 copying lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:04,444 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:04,446 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:04,448 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:04,450 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:04,452 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:04,455 copying lm_eval/tasks/include/few_shot_og/Albanian/_include_base_44_albanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:04,457 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:04,459 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:04,461 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:04,464 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:04,466 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:04,468 copying lm_eval/tasks/include/few_shot_og/Finnish/_include_base_44_finnish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:04,470 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:04,472 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:04,475 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:04,477 copying lm_eval/tasks/include/few_shot_og/Spanish/_include_base_44_spanish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:04,479 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:04,481 copying lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:04,483 copying lm_eval/tasks/include/few_shot_og/Nepali/_include_base_44_nepali.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:04,486 copying lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:04,488 copying lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:04,491 copying lm_eval/tasks/include/few_shot_og/Polish/_include_base_44_polish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:04,493 copying lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:04,495 copying lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:04,498 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:04,500 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:04,502 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:04,504 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:04,507 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:04,509 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:04,511 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:04,513 copying lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:04,515 copying lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:04,518 copying lm_eval/tasks/include/few_shot_og/Serbian/_include_base_44_serbian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:04,520 copying lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:04,522 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:04,524 copying lm_eval/tasks/include/few_shot_og/Portuguese/_include_base_44_portuguese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:04,526 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:04,528 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:04,530 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:04,532 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:04,534 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:04,536 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:04,539 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:04,541 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:04,543 copying lm_eval/tasks/include/few_shot_og/French/_include_base_44_french.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:04,545 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:04,547 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:04,549 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:04,551 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:04,553 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:04,555 copying lm_eval/tasks/include/few_shot_og/Uzbek/_include_base_44_uzbek.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:04,557 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:04,559 copying lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:04,561 copying lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:04,563 copying lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:04,565 copying lm_eval/tasks/include/few_shot_og/Malay/_include_base_44_malay.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:04,568 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:04,570 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:04,572 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:04,574 copying lm_eval/tasks/include/few_shot_og/Italian/_include_base_44_italian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:04,576 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:04,579 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:04,581 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:04,583 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:04,585 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:04,587 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:04,589 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:04,591 copying lm_eval/tasks/include/few_shot_og/Estonian/_include_base_44_estonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:04,593 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:04,595 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:04,598 copying lm_eval/tasks/include/few_shot_og/Bengali/_include_base_44_bengali.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:04,600 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:04,603 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:04,606 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:04,609 copying lm_eval/tasks/include/few_shot_og/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:04,611 copying lm_eval/tasks/include/few_shot_og/Georgian/_include_base_44_georgian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:04,613 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,616 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,618 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,620 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,622 copying lm_eval/tasks/include/few_shot_og/Russian/_include_base_44_russian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,624 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,626 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,628 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,631 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:04,633 copying lm_eval/tasks/include/few_shot_og/Korean/_include_base_44_korean.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:04,635 copying lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:04,637 copying lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:04,639 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,641 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,644 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,646 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,648 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,650 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,652 copying lm_eval/tasks/include/few_shot_og/Chinese/_include_base_44_chinese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,654 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,657 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:04,659 copying lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:04,661 copying lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:04,663 copying lm_eval/tasks/include/few_shot_og/Croatian/_include_base_44_croatian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:04,665 copying lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:04,667 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:04,669 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:04,672 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:04,674 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:04,676 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:04,678 copying lm_eval/tasks/include/few_shot_og/Dutch/_include_base_44_dutch.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:04,680 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,682 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,684 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,686 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,688 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,690 copying lm_eval/tasks/include/few_shot_og/Hindi/_include_base_44_hindi.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,692 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,694 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,697 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:04,699 copying lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:04,702 copying lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:04,704 copying lm_eval/tasks/include/few_shot_og/Tamil/_include_base_44_tamil.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:04,707 copying lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:04,708 copying lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:04,711 copying lm_eval/tasks/include/few_shot_og/Hungarian/_include_base_44_hungarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:04,713 copying lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:04,715 copying lm_eval/tasks/include/few_shot_og/Turkish/_include_base_44_turkish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:04,717 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:04,719 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:04,721 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:04,723 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:04,725 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,727 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,729 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,731 copying lm_eval/tasks/include/few_shot_og/Greek/_include_base_44_greek.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,734 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,737 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,739 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,741 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:04,744 copying lm_eval/tasks/include/few_shot_og/Armenian/_include_base_44_armenian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:04,746 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:04,748 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:04,750 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:04,751 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:04,753 copying lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:04,756 copying lm_eval/tasks/include/few_shot_og/Hebrew/_include_base_44_hebrew.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:04,758 copying lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:04,760 copying lm_eval/tasks/include/few_shot_og/German/include_base_44_german_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:04,762 copying lm_eval/tasks/include/few_shot_og/German/include_base_44_german_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:04,764 copying lm_eval/tasks/include/few_shot_og/German/include_base_44_german_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:04,766 copying lm_eval/tasks/include/few_shot_og/German/_include_base_44_german.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:04,768 copying lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:04,770 copying lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:04,772 copying lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:04,775 copying lm_eval/tasks/include/few_shot_og/Japanese/_include_base_44_japanese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:04,777 copying lm_eval/tasks/include/few_shot_og/Basque/include_base_44_basque_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:04,779 copying lm_eval/tasks/include/few_shot_og/Basque/_include_base_44_basque.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:04,781 copying lm_eval/tasks/include/few_shot_og/Bulgarian/_include_base_44_bulgarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:04,783 copying lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:04,785 copying lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:04,787 copying lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:04,789 copying lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,791 copying lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,793 copying lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,795 copying lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,797 copying lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,799 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,801 copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,803 copying lm_eval/tasks/blimp/npi_present_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,805 copying lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,808 copying lm_eval/tasks/blimp/npi_present_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,810 copying lm_eval/tasks/blimp/causative.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,812 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,815 copying lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,817 copying lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,819 copying lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,822 copying lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,824 copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,826 copying lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,829 copying lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,832 copying lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,834 copying lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,836 copying lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,839 copying lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,841 copying lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,843 copying lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,845 copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,847 copying lm_eval/tasks/blimp/adjunct_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,849 copying lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,851 copying lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,854 copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,856 copying lm_eval/tasks/blimp/only_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,858 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,860 copying lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,862 copying lm_eval/tasks/blimp/drop_argument.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,864 copying lm_eval/tasks/blimp/complex_NP_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,866 copying lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,868 copying lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,870 copying lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,872 copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,875 copying lm_eval/tasks/blimp/passive_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,877 copying lm_eval/tasks/blimp/wh_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,879 copying lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,881 copying lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,883 copying lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,885 copying lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,887 copying lm_eval/tasks/blimp/inchoative.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,889 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,891 copying lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,893 copying lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,895 copying lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,897 copying lm_eval/tasks/blimp/_blimp.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,900 copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,902 copying lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,904 copying lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,906 copying lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,908 copying lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,910 copying lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,913 copying lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,915 copying lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,917 copying lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,919 copying lm_eval/tasks/blimp/intransitive.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,921 copying lm_eval/tasks/blimp/passive_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,924 copying lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,926 copying lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,928 copying lm_eval/tasks/blimp/transitive.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,930 copying lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,932 copying lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,934 copying lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:04,936 copying lm_eval/tasks/mmmu/mmmu_accounting.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,938 copying lm_eval/tasks/mmmu/mmmu_art_theory.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,940 copying lm_eval/tasks/mmmu/_tech_and_engineering.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,942 copying lm_eval/tasks/mmmu/mmmu_music.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,944 copying lm_eval/tasks/mmmu/mmmu_basic_medical_science.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,946 copying lm_eval/tasks/mmmu/mmmu_math.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,949 copying lm_eval/tasks/mmmu/mmmu_materials.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,951 copying lm_eval/tasks/mmmu/mmmu_sociology.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,953 copying lm_eval/tasks/mmmu/mmmu_design.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,955 copying lm_eval/tasks/mmmu/mmmu_history.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,957 copying lm_eval/tasks/mmmu/mmmu_economics.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,959 copying lm_eval/tasks/mmmu/mmmu_electronics.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,961 copying lm_eval/tasks/mmmu/mmmu_chemistry.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,964 copying lm_eval/tasks/mmmu/_health_and_medicine.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,966 copying lm_eval/tasks/mmmu/_art_and_design.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,968 copying lm_eval/tasks/mmmu/mmmu_finance.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,970 copying lm_eval/tasks/mmmu/_humanities_and_social_sciences.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,972 copying lm_eval/tasks/mmmu/_mmmu.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,974 copying lm_eval/tasks/mmmu/mmmu_marketing.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,976 copying lm_eval/tasks/mmmu/mmmu_computer_science.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,978 copying lm_eval/tasks/mmmu/mmmu_physics.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,980 copying lm_eval/tasks/mmmu/mmmu_architecture_and_engineering.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,983 copying lm_eval/tasks/mmmu/mmmu_diagnostics_and_laboratory_medicine.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,985 copying lm_eval/tasks/mmmu/mmmu_biology.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,988 copying lm_eval/tasks/mmmu/mmmu_geography.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,991 copying lm_eval/tasks/mmmu/_science.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,994 copying lm_eval/tasks/mmmu/mmmu_clinical_medicine.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,997 copying lm_eval/tasks/mmmu/_business.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:04,999 copying lm_eval/tasks/mmmu/mmmu_art.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,001 copying lm_eval/tasks/mmmu/mmmu_energy_and_power.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,003 copying lm_eval/tasks/mmmu/mmmu_manage.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,005 copying lm_eval/tasks/mmmu/mmmu_public_health.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,008 copying lm_eval/tasks/mmmu/mmmu_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,010 copying lm_eval/tasks/mmmu/mmmu_literature.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,012 copying lm_eval/tasks/mmmu/mmmu_pharmacy.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,014 copying lm_eval/tasks/mmmu/mmmu_psychology.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,016 copying lm_eval/tasks/mmmu/mmmu_agriculture.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:05,018 copying lm_eval/tasks/tmlu/default/tmlu_AST_history.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,020 copying lm_eval/tasks/tmlu/default/tmlu_driving_rule.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,022 copying lm_eval/tasks/tmlu/default/tmlu_clinical_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,024 copying lm_eval/tasks/tmlu/default/tmlu_CAP_earth_science.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,027 copying lm_eval/tasks/tmlu/default/tmlu_CAP_civics.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,029 copying lm_eval/tasks/tmlu/default/tmlu_AST_geography.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,031 copying lm_eval/tasks/tmlu/default/tmlu_clinical_psychologist.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,033 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_geography.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,035 copying lm_eval/tasks/tmlu/default/tmlu_CAP_biology.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,037 copying lm_eval/tasks/tmlu/default/tmlu_teacher_qualification.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,039 copying lm_eval/tasks/tmlu/default/tmlu_basic_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,042 copying lm_eval/tasks/tmlu/default/tmlu_tour_leader.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,044 copying lm_eval/tasks/tmlu/default/tmlu_nutritionist.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,046 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_earth_science.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,049 copying lm_eval/tasks/tmlu/default/tmlu_AST_chinese.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,051 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_civics.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,053 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_biology.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,056 copying lm_eval/tasks/tmlu/default/tmlu_CAP_chinese.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,059 copying lm_eval/tasks/tmlu/default/tmlu_AST_biology.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,061 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_history.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,063 copying lm_eval/tasks/tmlu/default/tmlu_CAP_geography.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,066 copying lm_eval/tasks/tmlu/default/tmlu_tour_guide.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,068 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_chemistry.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,070 copying lm_eval/tasks/tmlu/default/tmlu_accountant.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,073 copying lm_eval/tasks/tmlu/default/tmlu_AST_chemistry.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,075 copying lm_eval/tasks/tmlu/default/_tmlu.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,077 copying lm_eval/tasks/tmlu/default/tmlu_AST_civics.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,080 copying lm_eval/tasks/tmlu/default/tmlu_CAP_chemistry.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,082 copying lm_eval/tasks/tmlu/default/tmlu_taiwan_tourist_resources.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,085 copying lm_eval/tasks/tmlu/default/tmlu_lawyer_qualification.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,087 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_chinese.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,089 copying lm_eval/tasks/tmlu/default/tmlu_CAP_history.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:05,091 copying lm_eval/tasks/egymmlu/egymmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,093 copying lm_eval/tasks/egymmlu/egymmlu_driving_test.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,095 copying lm_eval/tasks/egymmlu/egymmlu_biology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,097 copying lm_eval/tasks/egymmlu/egymmlu_philosophy_ar.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,100 copying lm_eval/tasks/egymmlu/egymmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,102 copying lm_eval/tasks/egymmlu/egymmlu_nutrition.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,104 copying lm_eval/tasks/egymmlu/egymmlu_political_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,106 copying lm_eval/tasks/egymmlu/egymmlu_history.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,108 copying lm_eval/tasks/egymmlu/egymmlu_social_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,109 copying lm_eval/tasks/egymmlu/egymmlu_geography.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,111 copying lm_eval/tasks/egymmlu/egymmlu_math.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,113 copying lm_eval/tasks/egymmlu/egymmlu_natural_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,115 copying lm_eval/tasks/egymmlu/egymmlu_economics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,117 copying lm_eval/tasks/egymmlu/egymmlu_law.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,119 copying lm_eval/tasks/egymmlu/egymmlu_civics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,122 copying lm_eval/tasks/egymmlu/egymmlu_public_relations.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,125 copying lm_eval/tasks/egymmlu/egymmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,128 copying lm_eval/tasks/egymmlu/egymmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,131 copying lm_eval/tasks/egymmlu/egymmlu_world_religions.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,134 copying lm_eval/tasks/egymmlu/egymmlu_computer_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,137 copying lm_eval/tasks/egymmlu/egymmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,139 copying lm_eval/tasks/egymmlu/egymmlu_arabic_language_(grammar).yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,141 copying lm_eval/tasks/egymmlu/_egymmlu.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,143 copying lm_eval/tasks/egymmlu/egymmlu_human_aging.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,145 copying lm_eval/tasks/egymmlu/egymmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,147 copying lm_eval/tasks/egymmlu/egymmlu_security_studies.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,149 copying lm_eval/tasks/egymmlu/egymmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,151 copying lm_eval/tasks/egymmlu/egymmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,153 copying lm_eval/tasks/egymmlu/egymmlu_physics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,155 copying lm_eval/tasks/egymmlu/egymmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,157 copying lm_eval/tasks/egymmlu/egymmlu_accounting.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,159 copying lm_eval/tasks/egymmlu/egymmlu_management_ar.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,161 copying lm_eval/tasks/egymmlu/egymmlu_international_law.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,163 copying lm_eval/tasks/egymmlu/egymmlu_arabic_language.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,165 copying lm_eval/tasks/egymmlu/_egymmlu_mmlu.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,167 copying lm_eval/tasks/egymmlu/egymmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,169 copying lm_eval/tasks/egymmlu/egymmlu_general_knowledge.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,171 copying lm_eval/tasks/egymmlu/egymmlu_islamic_studies.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,173 copying lm_eval/tasks/egymmlu/egymmlu_sociology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,175 copying lm_eval/tasks/egymmlu/_egymmlu_ar_mmlu.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,177 copying lm_eval/tasks/egymmlu/egymmlu_arabic_language_(general).yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,179 copying lm_eval/tasks/egymmlu/egymmlu_global_facts.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,181 copying lm_eval/tasks/egymmlu/egymmlu_management.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,183 copying lm_eval/tasks/egymmlu/egymmlu_philosophy.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,186 copying lm_eval/tasks/egymmlu/egymmlu_professional_law.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,188 copying lm_eval/tasks/egymmlu/egymmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,190 copying lm_eval/tasks/egymmlu/egymmlu_marketing.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:05,192 copying lm_eval/tasks/moral_stories/moral_stories.yaml -> build/lib/lm_eval/tasks/moral_stories 2026-05-11T14:29:05,194 creating build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:05,195 copying lm_eval/tasks/mastermind/mastermind_46_easy.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:05,197 copying lm_eval/tasks/mastermind/mastermind_46_hard.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:05,200 copying lm_eval/tasks/mastermind/mastermind_35_hard.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:05,202 copying lm_eval/tasks/mastermind/mastermind_24_hard.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:05,204 copying lm_eval/tasks/mastermind/mastermind_35_easy.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:05,206 copying lm_eval/tasks/mastermind/mastermind_24_easy.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:05,208 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10cm.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,210 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,212 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,214 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_atc.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,216 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,218 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,220 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,222 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9cm.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,224 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,226 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,228 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,230 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,232 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9proc.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,234 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,236 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,239 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10proc.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,241 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,243 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,245 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,247 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,249 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:05,251 copying lm_eval/tasks/babilong/babilong_qa3.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,253 copying lm_eval/tasks/babilong/babilong_qa7.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,255 copying lm_eval/tasks/babilong/babilong_qa16.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,257 copying lm_eval/tasks/babilong/babilong_qa5.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,260 copying lm_eval/tasks/babilong/babilong_qa12.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,262 copying lm_eval/tasks/babilong/babilong_qa1.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,264 copying lm_eval/tasks/babilong/babilong_qa18.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,266 copying lm_eval/tasks/babilong/babilong_longctx.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,268 copying lm_eval/tasks/babilong/babilong_qa14.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,270 copying lm_eval/tasks/babilong/babilong_qa10.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,272 copying lm_eval/tasks/babilong/babilong_qa2.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,274 copying lm_eval/tasks/babilong/babilong_qa8.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,277 copying lm_eval/tasks/babilong/babilong_qa17.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,279 copying lm_eval/tasks/babilong/babilong_qa11.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,281 copying lm_eval/tasks/babilong/babilong_qa9.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,283 copying lm_eval/tasks/babilong/babilong.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,285 copying lm_eval/tasks/babilong/babilong_qa20.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,288 copying lm_eval/tasks/babilong/babilong_qa15.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,290 copying lm_eval/tasks/babilong/babilong_qa4.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,292 copying lm_eval/tasks/babilong/babilong_qa19.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,294 copying lm_eval/tasks/babilong/babilong_qa13.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,296 copying lm_eval/tasks/babilong/babilong_qa6.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:05,298 creating build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,299 copying lm_eval/tasks/longbench2/legal_multi.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,301 copying lm_eval/tasks/longbench2/many_shot.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,303 copying lm_eval/tasks/longbench2/agent_history.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,305 copying lm_eval/tasks/longbench2/news_multi.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,307 copying lm_eval/tasks/longbench2/user_guide.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,309 copying lm_eval/tasks/longbench2/govt_multi_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,311 copying lm_eval/tasks/longbench2/academic_single.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,313 copying lm_eval/tasks/longbench2/lit_single_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,315 copying lm_eval/tasks/longbench2/academic_multi_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,317 copying lm_eval/tasks/longbench2/_longbench2_incontext.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,319 copying lm_eval/tasks/longbench2/_longbench2_multi.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,321 copying lm_eval/tasks/longbench2/_longbench2.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,323 copying lm_eval/tasks/longbench2/graph.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,325 copying lm_eval/tasks/longbench2/govt_single_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,326 copying lm_eval/tasks/longbench2/_longbench2_history.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,328 copying lm_eval/tasks/longbench2/legal_single.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,330 copying lm_eval/tasks/longbench2/table.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,332 copying lm_eval/tasks/longbench2/dialogue_history.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,335 copying lm_eval/tasks/longbench2/_longbench2_structured.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,337 copying lm_eval/tasks/longbench2/fin_single_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,339 copying lm_eval/tasks/longbench2/translate.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,341 copying lm_eval/tasks/longbench2/detective.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,343 copying lm_eval/tasks/longbench2/_longbench2_single.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,345 copying lm_eval/tasks/longbench2/fin_multi_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,347 copying lm_eval/tasks/longbench2/longbench2_code.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,349 copying lm_eval/tasks/longbench2/event_order.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:05,351 copying lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:05,353 copying lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:05,355 copying lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:05,358 copying lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:05,360 copying lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:05,362 copying lm_eval/tasks/code_x_glue/code-text/_codexglue.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:05,364 copying lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:05,366 creating build/lib/lm_eval/tasks/sciq 2026-05-11T14:29:05,367 copying lm_eval/tasks/sciq/sciq.yaml -> build/lib/lm_eval/tasks/sciq 2026-05-11T14:29:05,369 copying lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,372 copying lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,373 copying lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,375 copying lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,377 copying lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,379 copying lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,381 copying lm_eval/tasks/tinyBenchmarks/tinyArc.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,383 copying lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:05,385 copying lm_eval/tasks/chartqa/chartqa_llama_90.yaml -> build/lib/lm_eval/tasks/chartqa 2026-05-11T14:29:05,387 copying lm_eval/tasks/chartqa/chartqa_llama.yaml -> build/lib/lm_eval/tasks/chartqa 2026-05-11T14:29:05,389 copying lm_eval/tasks/chartqa/chartqa.yaml -> build/lib/lm_eval/tasks/chartqa 2026-05-11T14:29:05,391 copying lm_eval/tasks/minerva_math/minerva_math500.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,393 copying lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,395 copying lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,397 copying lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,399 copying lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,401 copying lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,403 copying lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,405 copying lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:05,407 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,409 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,411 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,414 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,416 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,418 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,420 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,422 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,425 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,427 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,429 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,431 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,433 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,435 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,438 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,440 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,442 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,444 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,447 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,449 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,451 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,454 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,456 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,458 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,461 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,463 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,465 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,468 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,470 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,472 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,474 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,476 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,479 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,481 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,483 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,485 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,487 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,490 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,492 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,494 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,496 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,499 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,501 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,503 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,505 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,507 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,509 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,511 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,514 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,516 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,518 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,520 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,522 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,524 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,527 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,529 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,532 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,534 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,536 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,538 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,541 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,543 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:05,545 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,548 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,552 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,555 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,557 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,560 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,562 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,564 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,567 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,570 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,572 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,575 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,577 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,579 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,582 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,585 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,587 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,590 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,593 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,595 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,598 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,600 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,603 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,606 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,609 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,611 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,614 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,616 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,619 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,622 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,624 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,626 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,629 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,631 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,634 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,636 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,639 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,641 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,644 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,646 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,649 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,652 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,654 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,656 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,659 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,661 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,664 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,666 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,669 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,671 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,674 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,676 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,678 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,681 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,684 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,687 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,689 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,692 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,695 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,698 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,701 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,703 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:05,706 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,709 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,711 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,713 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,715 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,717 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,720 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,722 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,724 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,726 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,729 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,731 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,733 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,735 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,737 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,740 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,742 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,745 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,747 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,749 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,751 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,754 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,756 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,758 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,760 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,762 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,765 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,767 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,769 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,772 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,774 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,776 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,778 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,780 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,783 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,785 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,787 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,790 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,792 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,794 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,796 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,799 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,801 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,803 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,805 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,807 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,809 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,812 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,814 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,816 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,818 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,821 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,823 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,825 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,828 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,830 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,832 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,835 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,837 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,839 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,842 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,844 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:05,847 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,849 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,852 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,854 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,856 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,859 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,862 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,864 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,867 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,869 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,872 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,874 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,877 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,879 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,881 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,883 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,885 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,887 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,890 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,892 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,894 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,897 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,899 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,901 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,903 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,906 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,908 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,911 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,913 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,915 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,918 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,920 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,923 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,925 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,928 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,930 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,933 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,935 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,938 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,940 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,942 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,944 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,946 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,948 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,951 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,953 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,955 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,957 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,959 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,961 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,964 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,966 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,968 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,971 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,973 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,975 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,977 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,980 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,983 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,985 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,987 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,989 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:05,991 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:05,994 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:05,996 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:05,998 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,000 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,003 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,005 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,008 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,011 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,013 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,015 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,018 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,020 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,022 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,024 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,027 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,029 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,031 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,033 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,035 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,037 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,039 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,041 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,044 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,047 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,049 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,051 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,054 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,056 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,059 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,061 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,064 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,067 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,069 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,071 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,073 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,075 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,078 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,080 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,082 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,085 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,087 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,089 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,091 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,093 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,096 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,098 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,100 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,103 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,105 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,107 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,110 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,112 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,114 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,116 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,118 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,120 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,123 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,125 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,127 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,130 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,132 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:06,134 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,135 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,137 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,139 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,141 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,143 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,145 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,147 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,150 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,152 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,154 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,157 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,159 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,162 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,164 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,166 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,168 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,170 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,172 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,174 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,176 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,178 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,180 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,183 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,185 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,188 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,190 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,192 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,194 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,196 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,198 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,200 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,202 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,204 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,206 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,208 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,210 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,212 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,214 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,216 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,218 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,220 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,222 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,224 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,226 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,228 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,230 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,232 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,234 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,237 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,239 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,241 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,243 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,245 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,247 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,249 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,251 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,253 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,255 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,257 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,259 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,261 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,264 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:06,266 copying lm_eval/tasks/llama3/instruct/arc_challenge/arc_challenge_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T14:29:06,268 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,270 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,272 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,275 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,277 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,279 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,282 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,284 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,286 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,289 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,291 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,294 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,296 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,299 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,301 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,303 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,305 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,307 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,310 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,312 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,314 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,316 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,318 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,320 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,322 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,324 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,327 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,329 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,332 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,334 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,336 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,338 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,340 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,343 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,345 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,348 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,350 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,352 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,355 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,357 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,359 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,361 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,363 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,365 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,368 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,370 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,373 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,375 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,377 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,379 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,382 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,384 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,386 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,389 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,391 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,393 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,395 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,398 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,400 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,402 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,404 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,407 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:06,409 creating build/lib/lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T14:29:06,410 copying lm_eval/tasks/llama3/instruct/gsm8k/gsm8k.yaml -> build/lib/lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T14:29:06,412 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,413 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,415 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,418 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,419 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,421 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,423 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,425 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,427 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,429 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,431 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,433 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,435 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,437 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,439 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,441 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,443 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,445 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,447 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,449 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,451 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,453 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,455 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,457 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,459 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,461 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,463 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,465 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,467 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,470 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,473 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,475 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,478 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,480 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,482 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,484 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,486 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,488 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,490 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,492 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,494 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,496 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,498 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,500 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,502 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,504 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,507 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,509 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,511 copying lm_eval/tasks/llama3/instruct/mmlu/llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,513 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,515 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,517 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,519 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,521 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,523 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,525 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,528 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,529 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,532 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,534 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,536 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,538 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,539 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:06,541 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_health.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,543 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,545 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,547 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,549 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,551 copying lm_eval/tasks/llama3/instruct/mmlu_pro/_mmlu_pro.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,553 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,555 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_math.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,557 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,559 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_economics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,561 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,563 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_business.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,565 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,567 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,569 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:06,571 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,573 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,576 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,578 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,581 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,583 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,586 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,589 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,591 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,594 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,596 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,599 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,602 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,604 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,607 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,609 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,612 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,614 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,617 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,620 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,623 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,625 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,627 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,630 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,633 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,636 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,639 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,641 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,644 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,647 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,649 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,652 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,655 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,657 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,660 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,662 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,665 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,668 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,670 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,672 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,675 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,677 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,679 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,682 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,684 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,687 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,689 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,692 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,694 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,697 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,700 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,703 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,705 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,708 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,710 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,713 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,715 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,718 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,721 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,724 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,726 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,729 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:06,731 creating build/lib/lm_eval/tasks/commonsense_qa 2026-05-11T14:29:06,732 copying lm_eval/tasks/commonsense_qa/default.yaml -> build/lib/lm_eval/tasks/commonsense_qa 2026-05-11T14:29:06,734 creating build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,735 copying lm_eval/tasks/zhoblimp/fci_renhe_prepP.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,737 copying lm_eval/tasks/zhoblimp/BA_verb_le_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,739 copying lm_eval/tasks/zhoblimp/preposition_insertion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,741 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_intran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,743 copying lm_eval/tasks/zhoblimp/npi_renhe_wh_question_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,744 copying lm_eval/tasks/zhoblimp/existential_there_subject_raising.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,746 copying lm_eval/tasks/zhoblimp/fci_renhe_dou.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,748 copying lm_eval/tasks/zhoblimp/BA_duplicate_argument.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,750 copying lm_eval/tasks/zhoblimp/question_particle_daodi_choice_intran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,752 copying lm_eval/tasks/zhoblimp/ellipsis_double_object.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,754 copying lm_eval/tasks/zhoblimp/BA_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,756 copying lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,758 copying lm_eval/tasks/zhoblimp/question_nandao_scope_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,760 copying lm_eval/tasks/zhoblimp/question_nandao_raising_1_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,762 copying lm_eval/tasks/zhoblimp/BEI_construction_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,764 copying lm_eval/tasks/zhoblimp/noun_phrase_conjunction_jian.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,766 copying lm_eval/tasks/zhoblimp/agent_causative.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,768 copying lm_eval/tasks/zhoblimp/left_adverbial_d.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,770 copying lm_eval/tasks/zhoblimp/principle_A_c_command_number.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,772 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,774 copying lm_eval/tasks/zhoblimp/question_particle_daodi_choice_tran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,776 copying lm_eval/tasks/zhoblimp/left_adverbial_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,778 copying lm_eval/tasks/zhoblimp/BA_verb_le_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,780 copying lm_eval/tasks/zhoblimp/adjective_transitive_dui.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,782 copying lm_eval/tasks/zhoblimp/question_nandao_raising_3.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,785 copying lm_eval/tasks/zhoblimp/BA_BEI_subj_drop.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,787 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,789 copying lm_eval/tasks/zhoblimp/question_nandao_raising_1_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,791 copying lm_eval/tasks/zhoblimp/fci_renhe_ruguo.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,793 copying lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_locP.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,796 copying lm_eval/tasks/zhoblimp/fci_renhe_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,798 copying lm_eval/tasks/zhoblimp/question_nandao_scope_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,802 copying lm_eval/tasks/zhoblimp/preposition_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,804 copying lm_eval/tasks/zhoblimp/intransitive_double_obj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,806 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,808 copying lm_eval/tasks/zhoblimp/topicalization_SOV.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,810 copying lm_eval/tasks/zhoblimp/ya_insertion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,812 copying lm_eval/tasks/zhoblimp/relative_operator_who.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,814 copying lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,816 copying lm_eval/tasks/zhoblimp/passive_no_adj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,818 copying lm_eval/tasks/zhoblimp/anaphor_gender_agreement.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,820 copying lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,822 copying lm_eval/tasks/zhoblimp/passive_intransitive.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,824 copying lm_eval/tasks/zhoblimp/principle_A_domain.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,826 copying lm_eval/tasks/zhoblimp/causative_shi_ba.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,829 copying lm_eval/tasks/zhoblimp/plural_cardinal_men_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,831 copying lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,833 copying lm_eval/tasks/zhoblimp/fci_renhe_suoyou.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,835 copying lm_eval/tasks/zhoblimp/BA_inversion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,837 copying lm_eval/tasks/zhoblimp/left_adverbial_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,839 copying lm_eval/tasks/zhoblimp/npi_renhe_wh_question_obj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,841 copying lm_eval/tasks/zhoblimp/question_A_not_A.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,843 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_tran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,845 copying lm_eval/tasks/zhoblimp/BEI_preposition.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,847 copying lm_eval/tasks/zhoblimp/classifier_noun_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,849 copying lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,851 copying lm_eval/tasks/zhoblimp/intransitive_no_obj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,853 copying lm_eval/tasks/zhoblimp/right_yijing_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,855 copying lm_eval/tasks/zhoblimp/PN_numP_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,857 copying lm_eval/tasks/zhoblimp/zhoblimp_group.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,859 copying lm_eval/tasks/zhoblimp/relative_operator_intepretation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,862 copying lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,864 copying lm_eval/tasks/zhoblimp/BEI_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,866 copying lm_eval/tasks/zhoblimp/right_yijing_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,868 copying lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,871 copying lm_eval/tasks/zhoblimp/modal_raising_topicalization.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,873 copying lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,875 copying lm_eval/tasks/zhoblimp/BA_meiba.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,877 copying lm_eval/tasks/zhoblimp/BA_no_progressive.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,880 copying lm_eval/tasks/zhoblimp/passive_body_part.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,882 copying lm_eval/tasks/zhoblimp/question_V_not_VP_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,884 copying lm_eval/tasks/zhoblimp/control_modal_vs_raising_modal.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,885 copying lm_eval/tasks/zhoblimp/you_yige.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,887 copying lm_eval/tasks/zhoblimp/question_A_not_A_indirect.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,889 copying lm_eval/tasks/zhoblimp/PN_numP_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,891 copying lm_eval/tasks/zhoblimp/BA_suo_adverbial_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,893 copying lm_eval/tasks/zhoblimp/BA_no_stative_verb.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,895 copying lm_eval/tasks/zhoblimp/modal_raising_hui.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,897 copying lm_eval/tasks/zhoblimp/BEI_construction_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,899 copying lm_eval/tasks/zhoblimp/principle_A_domain_number.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,901 copying lm_eval/tasks/zhoblimp/npi_renhe_A_not_A_question.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,903 copying lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,905 copying lm_eval/tasks/zhoblimp/verb_negation_particle.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,907 copying lm_eval/tasks/zhoblimp/question_A_not_A_daodi_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,909 copying lm_eval/tasks/zhoblimp/agent_animacy_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,911 copying lm_eval/tasks/zhoblimp/agent_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,913 copying lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,915 copying lm_eval/tasks/zhoblimp/classifier_noun_agreement_no_gap.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,917 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,919 copying lm_eval/tasks/zhoblimp/ellipsis_adj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,921 copying lm_eval/tasks/zhoblimp/left_dou.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,923 copying lm_eval/tasks/zhoblimp/question_A_not_A_daodi_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,925 copying lm_eval/tasks/zhoblimp/question_nandao_raising_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,927 copying lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,929 copying lm_eval/tasks/zhoblimp/nominal_definite_men.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,932 copying lm_eval/tasks/zhoblimp/agent_animacy_adv.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,934 copying lm_eval/tasks/zhoblimp/anaphor_number_agreement.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,936 copying lm_eval/tasks/zhoblimp/left_adverbial_e.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,939 copying lm_eval/tasks/zhoblimp/passive_suo.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,941 copying lm_eval/tasks/zhoblimp/BA_suo_adverbial_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,943 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_long_left.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,945 copying lm_eval/tasks/zhoblimp/question_particle_nandao.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,947 copying lm_eval/tasks/zhoblimp/question_daodi_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,949 copying lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,951 copying lm_eval/tasks/zhoblimp/classifier_noun_agreement.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,953 copying lm_eval/tasks/zhoblimp/principle_A_c_command.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,956 copying lm_eval/tasks/zhoblimp/question_nandao_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,958 copying lm_eval/tasks/zhoblimp/question_V_not_VP_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,961 copying lm_eval/tasks/zhoblimp/nominal_modal_insertion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,964 copying lm_eval/tasks/zhoblimp/topicalization_OSV.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,966 copying lm_eval/tasks/zhoblimp/plural_cardinal_men_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,968 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_short.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,971 copying lm_eval/tasks/zhoblimp/agent_animacy_passive.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,972 copying lm_eval/tasks/zhoblimp/BA_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,975 copying lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,977 copying lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,979 copying lm_eval/tasks/zhoblimp/npi_renhe_conditional.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,981 copying lm_eval/tasks/zhoblimp/noun_adjective_shi.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,983 copying lm_eval/tasks/zhoblimp/ellipsis_n_bar_class.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:06,986 copying lm_eval/tasks/leaderboard/leaderboard.yaml -> build/lib/lm_eval/tasks/leaderboard 2026-05-11T14:29:06,988 copying lm_eval/tasks/leaderboard/musr/musr_team_allocation.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:06,990 copying lm_eval/tasks/leaderboard/musr/musr_object_placements.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:06,992 copying lm_eval/tasks/leaderboard/musr/musr_murder_mysteries.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:06,995 copying lm_eval/tasks/leaderboard/musr/_musr.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:06,997 creating build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:06,998 copying lm_eval/tasks/leaderboard/bbh_mc/object_counting.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,001 copying lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,003 copying lm_eval/tasks/leaderboard/bbh_mc/hyperbaton.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,005 copying lm_eval/tasks/leaderboard/bbh_mc/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,007 copying lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,009 copying lm_eval/tasks/leaderboard/bbh_mc/snarks.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,012 copying lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,014 copying lm_eval/tasks/leaderboard/bbh_mc/web_of_lies.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,016 copying lm_eval/tasks/leaderboard/bbh_mc/causal_judgement.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,018 copying lm_eval/tasks/leaderboard/bbh_mc/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,021 copying lm_eval/tasks/leaderboard/bbh_mc/boolean_expressions.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,023 copying lm_eval/tasks/leaderboard/bbh_mc/temporal_sequences.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,025 copying lm_eval/tasks/leaderboard/bbh_mc/sports_understanding.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,027 copying lm_eval/tasks/leaderboard/bbh_mc/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,029 copying lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,031 copying lm_eval/tasks/leaderboard/bbh_mc/movie_recommendation.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,033 copying lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,036 copying lm_eval/tasks/leaderboard/bbh_mc/navigate.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,038 copying lm_eval/tasks/leaderboard/bbh_mc/ruin_names.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,040 copying lm_eval/tasks/leaderboard/bbh_mc/formal_fallacies.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,042 copying lm_eval/tasks/leaderboard/bbh_mc/date_understanding.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,044 copying lm_eval/tasks/leaderboard/bbh_mc/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,047 copying lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,049 copying lm_eval/tasks/leaderboard/bbh_mc/geometric_shapes.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,052 copying lm_eval/tasks/leaderboard/bbh_mc/_leaderboard_bbh.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:07,054 copying lm_eval/tasks/leaderboard/gpqa/gpqa_main_zeroshot.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:07,056 copying lm_eval/tasks/leaderboard/gpqa/gpqa_extended_zeroshot.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:07,058 copying lm_eval/tasks/leaderboard/gpqa/_leaderboard_gpqa.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:07,061 copying lm_eval/tasks/leaderboard/gpqa/gpqa_diamond_zeroshot.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:07,063 copying lm_eval/tasks/leaderboard/ifeval/ifeval.yaml -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:07,065 copying lm_eval/tasks/leaderboard/ifeval/_leaderboard_instruction_following.yaml -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:07,067 copying lm_eval/tasks/leaderboard/math/_leaderboard_math.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,069 copying lm_eval/tasks/leaderboard/math/math_geometry.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,071 copying lm_eval/tasks/leaderboard/math/math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,073 copying lm_eval/tasks/leaderboard/math/math_num_theory.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,075 copying lm_eval/tasks/leaderboard/math/math_algebra.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,078 copying lm_eval/tasks/leaderboard/math/math_precalculus.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,080 copying lm_eval/tasks/leaderboard/math/math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,082 copying lm_eval/tasks/leaderboard/math/math_prealgebra.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:07,084 copying lm_eval/tasks/leaderboard/mmlu_pro/mmlu_pro.yaml -> build/lib/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T14:29:07,086 copying lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:07,088 copying lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:07,090 copying lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:07,092 copying lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:07,094 copying lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:07,095 copying lm_eval/tasks/squad_completion/squad_completion.yaml -> build/lib/lm_eval/tasks/squad_completion 2026-05-11T14:29:07,097 creating build/lib/lm_eval/tasks/triviaqa 2026-05-11T14:29:07,098 copying lm_eval/tasks/triviaqa/default.yaml -> build/lib/lm_eval/tasks/triviaqa 2026-05-11T14:29:07,101 copying lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,102 copying lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,104 copying lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,107 copying lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,108 copying lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,110 copying lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,113 copying lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,115 copying lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,117 copying lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,119 copying lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,121 copying lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,123 copying lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,125 copying lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,127 copying lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,129 copying lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,131 copying lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,133 copying lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,135 copying lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,137 copying lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,140 copying lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,142 copying lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,144 copying lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,146 copying lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,148 copying lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,151 copying lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,153 copying lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,155 copying lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,157 copying lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,159 copying lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,161 copying lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,164 copying lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,166 copying lm_eval/tasks/belebele/_belebele.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,168 copying lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,170 copying lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,172 copying lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,174 copying lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,176 copying lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,178 copying lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,180 copying lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,182 copying lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,184 copying lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,186 copying lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,188 copying lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,190 copying lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,192 copying lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,194 copying lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,197 copying lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,199 copying lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,201 copying lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,203 copying lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,205 copying lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,207 copying lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,209 copying lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,211 copying lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,213 copying lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,215 copying lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,217 copying lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,219 copying lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,221 copying lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,224 copying lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,226 copying lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,228 copying lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,230 copying lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,232 copying lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,234 copying lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,236 copying lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,238 copying lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,239 copying lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,241 copying lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,243 copying lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,245 copying lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,247 copying lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,249 copying lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,251 copying lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,253 copying lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,255 copying lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,257 copying lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,259 copying lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,261 copying lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,263 copying lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,265 copying lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,267 copying lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,269 copying lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,271 copying lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,273 copying lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,275 copying lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,277 copying lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,279 copying lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,281 copying lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,283 copying lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,285 copying lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,288 copying lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,290 copying lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,292 copying lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,294 copying lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,296 copying lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,298 copying lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,300 copying lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,302 copying lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,304 copying lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,306 copying lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,308 copying lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,310 copying lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,312 copying lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,314 copying lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,317 copying lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,319 copying lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,322 copying lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,324 copying lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,326 copying lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,328 copying lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,330 copying lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,332 copying lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,334 copying lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,336 copying lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,338 copying lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,340 copying lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,342 copying lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,344 copying lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,347 copying lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,349 copying lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,351 copying lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,353 copying lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:07,355 copying lm_eval/tasks/basque_bench/arc_eu_easy.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,357 copying lm_eval/tasks/basque_bench/paws_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,359 copying lm_eval/tasks/basque_bench/xcopa_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,361 copying lm_eval/tasks/basque_bench/arc_eu_challenge.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,364 copying lm_eval/tasks/basque_bench/mgsm_native_cot_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,366 copying lm_eval/tasks/basque_bench/wnli_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,368 copying lm_eval/tasks/basque_bench/piqa_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,370 copying lm_eval/tasks/basque_bench/basque_bench.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,372 copying lm_eval/tasks/basque_bench/mgsm_direct_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:07,374 copying lm_eval/tasks/basque_bench/flores_eu/flores_ca-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,376 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-es.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,378 copying lm_eval/tasks/basque_bench/flores_eu/flores_pt-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,380 copying lm_eval/tasks/basque_bench/flores_eu/flores_de-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,382 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-it.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,384 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-ca.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,386 copying lm_eval/tasks/basque_bench/flores_eu/flores_gl-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,388 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-gl.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,390 copying lm_eval/tasks/basque_bench/flores_eu/flores_fr-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,391 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-en.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,393 copying lm_eval/tasks/basque_bench/flores_eu/flores_es-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,395 copying lm_eval/tasks/basque_bench/flores_eu/flores_en-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,397 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-fr.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,399 copying lm_eval/tasks/basque_bench/flores_eu/flores_it-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,402 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-pt.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,404 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-de.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,406 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:07,408 copying lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:29:07,410 copying lm_eval/tasks/kobest/kobest_copa.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:29:07,412 copying lm_eval/tasks/kobest/_kobest.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:29:07,414 copying lm_eval/tasks/kobest/kobest_boolq.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:29:07,416 copying lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:29:07,418 copying lm_eval/tasks/kobest/kobest_wic.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:29:07,420 creating build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,421 copying lm_eval/tasks/inverse_scaling/inverse_scaling_repetitive_algebra.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,423 copying lm_eval/tasks/inverse_scaling/inverse_scaling_sig_figs.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,426 copying lm_eval/tasks/inverse_scaling/inverse_scaling_redefine_math.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,428 copying lm_eval/tasks/inverse_scaling/inverse_scaling_modus_tollens.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,430 copying lm_eval/tasks/inverse_scaling/inverse_scaling_neqa.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,432 copying lm_eval/tasks/inverse_scaling/inverse_scaling_pattern_matching_suppression.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,434 copying lm_eval/tasks/inverse_scaling/inverse_scaling_quote_repetition.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,437 copying lm_eval/tasks/inverse_scaling/inverse_scaling_memo_trap.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,439 copying lm_eval/tasks/inverse_scaling/inverse_scaling_winobias_antistereotype.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,441 copying lm_eval/tasks/inverse_scaling/inverse_scaling_into_the_unknown.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,443 copying lm_eval/tasks/inverse_scaling/inverse_scaling_hindsight_neglect.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:07,445 copying lm_eval/tasks/logiqa/logiqa.yaml -> build/lib/lm_eval/tasks/logiqa 2026-05-11T14:29:07,447 copying lm_eval/tasks/mutual/mutual.yaml -> build/lib/lm_eval/tasks/mutual 2026-05-11T14:29:07,449 copying lm_eval/tasks/mutual/multual_plus.yaml -> build/lib/lm_eval/tasks/mutual 2026-05-11T14:29:07,452 copying lm_eval/tasks/galician_bench/xstorycloze_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,454 copying lm_eval/tasks/galician_bench/summarization_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,456 copying lm_eval/tasks/galician_bench/openbookqa_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,458 copying lm_eval/tasks/galician_bench/paws_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,460 copying lm_eval/tasks/galician_bench/parafrases_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,462 copying lm_eval/tasks/galician_bench/truthfulqa_gl_mc2.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,464 copying lm_eval/tasks/galician_bench/truthfulqa_gl_gen.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,466 copying lm_eval/tasks/galician_bench/galcola.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,468 copying lm_eval/tasks/galician_bench/xnli_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,470 copying lm_eval/tasks/galician_bench/mgsm_direct_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,472 copying lm_eval/tasks/galician_bench/truthfulqa_gl_mc1.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,474 copying lm_eval/tasks/galician_bench/belebele_glg_Latn.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,476 copying lm_eval/tasks/galician_bench/galician_bench.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:07,478 copying lm_eval/tasks/galician_bench/flores_gl/flores_fr-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,480 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-pt.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,482 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-eu.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,484 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-en.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,486 copying lm_eval/tasks/galician_bench/flores_gl/flores_it-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,488 copying lm_eval/tasks/galician_bench/flores_gl/flores_eu-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,490 copying lm_eval/tasks/galician_bench/flores_gl/flores_es-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,492 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-it.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,494 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-fr.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,496 copying lm_eval/tasks/galician_bench/flores_gl/flores_en-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,498 copying lm_eval/tasks/galician_bench/flores_gl/flores_ca-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,500 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-es.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,503 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,504 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-de.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,506 copying lm_eval/tasks/galician_bench/flores_gl/flores_pt-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,509 copying lm_eval/tasks/galician_bench/flores_gl/flores_de-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,510 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-ca.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:07,512 copying lm_eval/tasks/xcopa/default_zh.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,514 copying lm_eval/tasks/xcopa/default_ta.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,516 copying lm_eval/tasks/xcopa/default_ht.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,518 copying lm_eval/tasks/xcopa/default_it.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,520 copying lm_eval/tasks/xcopa/default_qu.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,523 copying lm_eval/tasks/xcopa/default_tr.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,525 copying lm_eval/tasks/xcopa/_xcopa.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,528 copying lm_eval/tasks/xcopa/default_et.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,530 copying lm_eval/tasks/xcopa/default_vi.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,532 copying lm_eval/tasks/xcopa/default_sw.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,535 copying lm_eval/tasks/xcopa/default_th.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,537 copying lm_eval/tasks/xcopa/default_id.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:07,539 copying lm_eval/tasks/cnn_dailymail/cnn_dailymail.yaml -> build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T14:29:07,541 copying lm_eval/tasks/ifeval/ifeval.yaml -> build/lib/lm_eval/tasks/ifeval 2026-05-11T14:29:07,543 copying lm_eval/tasks/ifeval/multilingual/ifeval_es.yaml -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T14:29:07,545 copying lm_eval/tasks/ifeval/multilingual/ifeval_ca.yaml -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T14:29:07,547 copying lm_eval/tasks/aradice/aradice.yaml -> build/lib/lm_eval/tasks/aradice 2026-05-11T14:29:07,550 copying lm_eval/tasks/aradice/winogrande/winogrande_egy.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:07,552 copying lm_eval/tasks/aradice/winogrande/winogrande_lev.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:07,554 copying lm_eval/tasks/aradice/winogrande/winogrande_msa.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:07,556 copying lm_eval/tasks/aradice/winogrande/winogrande_eng.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:07,558 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_msa.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:07,560 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_lev.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:07,563 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_egy.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:07,565 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_eng.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:07,567 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,569 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,571 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,573 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,575 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,577 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,579 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,581 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,583 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,585 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,587 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,590 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,592 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,594 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,595 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,598 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,600 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,601 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,604 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,605 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,608 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,610 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,612 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,614 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,616 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,618 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,620 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,622 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,624 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,626 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,628 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,630 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,632 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,634 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,636 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,638 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,640 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,642 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,644 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,646 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,648 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:07,650 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,652 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,654 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,656 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,658 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,660 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,663 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,665 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,667 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,669 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,671 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,673 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,675 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,677 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,679 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,681 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,683 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,685 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,687 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,689 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,691 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,693 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,695 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,697 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,699 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,701 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,703 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,705 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,707 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,709 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,712 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,714 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,715 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,717 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,719 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,721 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,723 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,725 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,727 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,729 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,731 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:07,733 copying lm_eval/tasks/aradice/cultural-benchmark/egypt.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:07,735 copying lm_eval/tasks/aradice/cultural-benchmark/syria.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:07,737 copying lm_eval/tasks/aradice/cultural-benchmark/lebanon.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:07,739 copying lm_eval/tasks/aradice/cultural-benchmark/palestine.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:07,741 copying lm_eval/tasks/aradice/cultural-benchmark/qatar.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:07,743 copying lm_eval/tasks/aradice/cultural-benchmark/jordan.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:07,745 copying lm_eval/tasks/aradice/piqa/piqa_eng.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T14:29:07,747 copying lm_eval/tasks/aradice/piqa/piqa_egy.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T14:29:07,749 copying lm_eval/tasks/aradice/piqa/piqa_msa.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T14:29:07,751 copying lm_eval/tasks/aradice/piqa/piqa_lev.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T14:29:07,753 copying lm_eval/tasks/aradice/openbookqa/openbookqa_egy.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:07,754 copying lm_eval/tasks/aradice/openbookqa/openbookqa_lev.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:07,756 copying lm_eval/tasks/aradice/openbookqa/openbookqa_eng.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:07,759 copying lm_eval/tasks/aradice/openbookqa/openbookqa_msa.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:07,761 copying lm_eval/tasks/aradice/boolq/ENG/boolq_eng.yaml -> build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:29:07,763 copying lm_eval/tasks/aradice/boolq/LEV/boolq_lev.yaml -> build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:29:07,765 copying lm_eval/tasks/aradice/boolq/MSA/boolq_msa.yaml -> build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:29:07,767 copying lm_eval/tasks/aradice/boolq/EGY/boolq_egy.yaml -> build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:29:07,769 creating build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,769 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,772 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,773 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,775 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,777 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,779 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,781 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:07,783 copying lm_eval/tasks/mediqa_qa2019/mediqa_qa2019.yaml -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:07,785 copying lm_eval/tasks/mediqa_qa2019/mediqa_qa2019_perplexity.yaml -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:07,787 copying lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:07,790 copying lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:07,792 copying lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:07,794 copying lm_eval/tasks/csatqa/_csatqa.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:07,796 copying lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:07,798 copying lm_eval/tasks/csatqa/csatqa_li.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:07,800 copying lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:07,802 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,804 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,806 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,808 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,810 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,812 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,814 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,817 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,819 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,821 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,823 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,825 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,827 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,829 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,831 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,833 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,835 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,837 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,839 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,841 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,843 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,845 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,847 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,849 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,851 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,854 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,856 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,858 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,860 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,862 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:07,864 copying lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,866 copying lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,868 copying lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,870 copying lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,872 copying lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,874 copying lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,876 copying lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,878 copying lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,880 copying lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,882 copying lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,884 copying lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,886 copying lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,888 copying lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,890 copying lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,892 copying lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,894 copying lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,896 copying lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,898 copying lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,900 copying lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,902 copying lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,904 copying lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,906 copying lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,908 copying lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,910 copying lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,912 copying lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,914 copying lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,916 copying lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,918 copying lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,920 copying lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,922 copying lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,924 copying lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:07,926 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,928 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,930 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,932 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,934 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,936 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,938 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,940 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,942 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,944 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,946 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,948 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,950 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,952 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,954 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,956 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,958 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,960 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,962 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,964 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,965 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,967 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,969 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,971 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,973 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,975 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,977 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,979 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,981 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,983 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,985 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,987 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,990 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,992 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:07,994 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:07,997 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:07,999 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,001 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,003 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,005 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,008 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,010 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,012 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,014 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,017 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,019 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,021 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,023 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,025 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,027 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,029 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,031 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,033 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,035 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,037 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,039 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,041 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,043 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,045 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,047 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,049 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,051 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,053 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,056 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,058 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,060 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,063 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,065 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,067 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,069 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,071 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,073 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,075 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,077 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,079 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,081 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,083 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,085 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,087 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,090 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,092 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,094 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,096 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,098 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,100 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,102 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,104 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,107 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,109 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,111 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,113 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,116 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,118 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,120 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,122 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,124 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:08,127 copying lm_eval/tasks/graphwalks/graphwalks_128k.yaml -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T14:29:08,129 copying lm_eval/tasks/graphwalks/graphwalks.yaml -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T14:29:08,131 copying lm_eval/tasks/graphwalks/graphwalks_1M.yaml -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T14:29:08,133 creating build/lib/lm_eval/tasks/fld 2026-05-11T14:29:08,134 copying lm_eval/tasks/fld/fld_logical_formula_default.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T14:29:08,136 copying lm_eval/tasks/fld/fld_default.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T14:29:08,138 copying lm_eval/tasks/fld/fld_star.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T14:29:08,140 copying lm_eval/tasks/fld/fld_logical_formula_star.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T14:29:08,142 creating build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:08,143 copying lm_eval/tasks/aexams/aexams_Social.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:08,146 copying lm_eval/tasks/aexams/aexams_Physics.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:08,148 copying lm_eval/tasks/aexams/aexams_Biology.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:08,150 copying lm_eval/tasks/aexams/aexams_Science.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:08,152 copying lm_eval/tasks/aexams/_aexams.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:08,154 copying lm_eval/tasks/aexams/aexams_IslamicStudies.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:08,157 copying lm_eval/tasks/afrixnli/direct/afrixnli.yaml -> build/lib/lm_eval/tasks/afrixnli/direct 2026-05-11T14:29:08,159 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,161 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,163 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,165 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,167 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,169 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,171 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,173 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,175 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,178 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,180 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,182 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,184 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,186 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,188 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,190 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,192 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,194 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:08,196 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,198 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,199 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,201 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,203 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,205 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,207 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,209 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,211 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,213 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,215 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,217 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,219 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,221 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,223 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,225 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,227 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,229 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:08,231 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,233 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,235 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,238 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,240 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,242 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,244 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,246 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,248 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,249 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,251 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,253 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,255 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,257 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,259 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,261 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,263 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,265 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:08,267 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,270 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,272 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,274 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,276 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,278 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,279 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,281 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,283 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,285 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,287 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,290 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,292 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,294 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,296 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,298 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,300 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,302 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:08,304 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,306 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,308 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,310 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,312 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,314 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,316 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,318 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,320 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,322 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,324 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,326 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,328 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,330 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,332 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,334 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,336 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,338 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:08,340 copying lm_eval/tasks/afrixnli/translate/afrixnli_tt.yaml -> build/lib/lm_eval/tasks/afrixnli/translate 2026-05-11T14:29:08,342 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,344 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,346 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,349 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,351 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,353 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,355 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,357 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,359 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,361 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,363 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,364 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,366 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,368 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,370 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,372 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,374 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:08,376 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,378 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,380 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,382 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,384 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,386 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,388 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,390 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,392 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,394 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,396 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,398 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,400 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,402 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,404 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,406 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,408 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:08,410 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,412 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,414 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,416 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,418 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,420 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,422 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,424 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,426 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,428 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,430 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,432 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,434 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,437 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,439 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,441 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,443 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:08,445 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,448 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,450 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,452 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,454 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,456 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,458 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,460 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,462 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,464 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,466 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,468 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,470 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,472 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,474 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,476 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,478 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:08,480 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,482 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,484 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,486 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,488 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,490 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,492 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,494 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,497 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,499 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,501 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,503 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,505 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,507 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,510 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,512 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,514 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:08,516 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,518 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,520 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,522 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,524 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,526 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,528 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,530 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,532 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,534 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,536 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,538 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,540 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,542 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,545 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,547 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,549 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,551 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:08,553 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,555 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,557 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,560 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,562 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,564 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,566 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,568 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,570 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,572 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,574 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,576 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,578 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,580 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,582 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,584 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,586 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:08,588 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,591 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,593 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,595 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,598 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,600 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,602 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,605 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,607 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,609 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,611 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,613 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,615 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,618 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,620 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,621 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,623 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,625 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:08,627 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,629 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,631 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,633 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,635 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,637 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,639 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,641 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,643 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,646 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,648 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,650 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,652 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,654 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,656 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,658 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,660 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,662 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:08,665 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,667 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,669 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,671 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,673 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,675 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,677 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,679 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,681 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,683 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,685 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,687 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,689 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,691 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,693 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,695 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,697 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:08,700 creating build/lib/lm_eval/tasks/wmdp 2026-05-11T14:29:08,701 copying lm_eval/tasks/wmdp/wmdp_chem.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T14:29:08,703 copying lm_eval/tasks/wmdp/wmdp_bio.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T14:29:08,705 copying lm_eval/tasks/wmdp/_wmdp.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T14:29:08,707 copying lm_eval/tasks/wmdp/wmdp_cyber.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T14:29:08,709 copying lm_eval/tasks/c4/c4.yaml -> build/lib/lm_eval/tasks/c4 2026-05-11T14:29:08,711 creating build/lib/lm_eval/tasks/bangla 2026-05-11T14:29:08,712 copying lm_eval/tasks/bangla/bangla_mmlu.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T14:29:08,714 copying lm_eval/tasks/bangla/bangla_openbookqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T14:29:08,716 copying lm_eval/tasks/bangla/bangla_commonsenseqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T14:29:08,719 copying lm_eval/tasks/bangla/bangla_boolqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T14:29:08,720 copying lm_eval/tasks/bangla/bangla_piqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T14:29:08,723 copying lm_eval/tasks/toksuite/_toksuite_all.yaml -> build/lib/lm_eval/tasks/toksuite 2026-05-11T14:29:08,725 copying lm_eval/tasks/toksuite/_toksuite_abridged.yaml -> build/lib/lm_eval/tasks/toksuite 2026-05-11T14:29:08,727 creating build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,728 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_spelled_out.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,730 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_equivalent_expressions.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,732 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_base.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,734 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_canonical.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,736 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_traditional.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,738 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_romanization.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,740 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_colloquial.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,742 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_keyboard_proximity_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,744 copying lm_eval/tasks/toksuite/toksuite_chinese/_toksuite_chinese.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,746 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_ocr_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,748 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_code_language_script_switching.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,750 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_word_spacing_zero-width_characters_extra_space.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,752 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_space_removal.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,754 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_partially_romanized.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,756 copying lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_optional_diacritics.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:08,758 creating build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,759 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_colloquial.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,761 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_base.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,763 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_keyboard_proximity_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,765 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_dialects.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,768 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_number_romanization.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,770 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_arabic_keyboard_for_farsi.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,772 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_equivalent_expressions.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,774 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_spelled_out.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,776 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_code_language_script_switching.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,778 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_word_spacing_zero-width_characters_extra_space.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,780 copying lm_eval/tasks/toksuite/toksuite_farsi/_toksuite_farsi.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,782 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_canonical.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,784 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_romanization.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,786 copying lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_optional_diacritics.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:08,788 creating build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,789 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_latex.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,791 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_chinese.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,793 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_spelled_out.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,796 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_farsi.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,798 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_italian.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,800 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_decorative_unicode.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,802 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_turkish.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,804 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_base.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,806 copying lm_eval/tasks/toksuite/toksuite_math/_toksuite_math.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,808 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_space_removal.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,810 copying lm_eval/tasks/toksuite/toksuite_math/toksuite_math_canonical.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:08,812 creating build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,813 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_plausible_diacritics_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,815 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_orthographic_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,817 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_code_language_script_switching.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,819 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_keyboard_proximity_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,822 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_base.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,824 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_similar_words.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,826 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_web_search_query.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,828 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_date_formats.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,830 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_contractions.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,832 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_grammatical_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,834 copying lm_eval/tasks/toksuite/toksuite_italian/_toksuite_italian.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,836 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_numerical_formats.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,838 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_typographical_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,840 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_dialects.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,842 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_english_keyboard.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,844 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_phonetic_spelling.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,846 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_capitalization.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,848 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_spelled_out.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,850 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_canonical.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,852 copying lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_abbreviations.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:08,854 creating build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,855 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_morpheme_separation.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,857 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_typographical_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,859 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_unicode_formatting.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,861 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_strikethrough.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,863 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_colloquial.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,865 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_fullwidth_characters.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,867 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_character_deletion.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,870 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_scripted_text.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,872 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_spelled_out.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,874 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_enclosed_characters.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,876 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_double_struck.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,878 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_upside_down_rotated.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,880 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_base.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,882 copying lm_eval/tasks/toksuite/toksuite_stem/_toksuite_stem.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,884 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_unusual_formatting.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,886 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_diacriticized_styling.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,888 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_superscript_subscript.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,890 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_space_removal.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,892 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_equivalent_expressions.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,894 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_canonical.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,896 copying lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_latex.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:08,898 creating build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,899 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_spelled_out.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,901 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_similar_words.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,903 copying lm_eval/tasks/toksuite/toksuite_turkish/_toksuite_turkish.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,905 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_dialects.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,908 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_grammatical_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,910 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_canonical.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,912 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_derivations.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,914 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_keyboard_proximity_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,916 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_code_language_script_switching.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,918 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_base.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,920 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_inflections.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,922 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_english_keyboard.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,924 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_colloquial.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,926 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_word_reordering.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,928 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_equivalent_expressions.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,930 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_date_formats.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,932 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_typographical_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,935 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_web_search_query.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,937 copying lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_orthographic_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:08,939 creating build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,940 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_emoji_substitution.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,942 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_date_formats.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,944 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_spaced_styling.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,946 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_homoglyphs.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,948 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_word_reordering.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,950 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_space_removal.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,952 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_letter_repetition_for_emphasis.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,954 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_contractions.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,956 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_lowercase.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,958 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_abbreviations.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,960 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_compounds.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,962 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_inflections.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,964 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_colloquial.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,967 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_ocr_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,969 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_scripted_text.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,971 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_superscript_subscript_styling.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,973 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_hyphenated_spelling.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,975 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_base.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,977 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_orthographic_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,980 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_character_deletion.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,982 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_character_substitution.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,984 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_web_search_query.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,986 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_historical_spelling.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,988 copying lm_eval/tasks/toksuite/toksuite_english/_toksuite_english.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,990 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_capitalization.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,992 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_keyboard_proximity_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,994 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_canonical.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,997 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_spelled_out.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:08,999 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_similar_words.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:09,001 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_grammatical_errors.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:09,003 copying lm_eval/tasks/toksuite/toksuite_english/toksuite_english_macron_diacritic.yaml -> build/lib/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:09,005 copying lm_eval/tasks/jfinqa/jfinqa_numerical.yaml -> build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:29:09,007 copying lm_eval/tasks/jfinqa/jfinqa_consistency.yaml -> build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:29:09,009 copying lm_eval/tasks/jfinqa/jfinqa_temporal.yaml -> build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:29:09,012 copying lm_eval/tasks/jfinqa/_jfinqa.yaml -> build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:29:09,014 creating build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:09,015 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-zeroshot.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:09,017 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-self-consistency.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:09,019 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-llama.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:09,021 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:09,023 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:09,025 creating build/lib/lm_eval/tasks/piqa 2026-05-11T14:29:09,026 copying lm_eval/tasks/piqa/piqa.yaml -> build/lib/lm_eval/tasks/piqa 2026-05-11T14:29:09,028 copying lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/lib/lm_eval/tasks/wmt2016 2026-05-11T14:29:09,030 creating build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,031 copying lm_eval/tasks/mela/mela_it.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,034 copying lm_eval/tasks/mela/mela_ru.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,036 copying lm_eval/tasks/mela/mela_de.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,039 copying lm_eval/tasks/mela/mela_es.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,041 copying lm_eval/tasks/mela/mela_ar.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,043 copying lm_eval/tasks/mela/mela_fr.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,045 copying lm_eval/tasks/mela/mela_en.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,047 copying lm_eval/tasks/mela/mela_is.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,050 copying lm_eval/tasks/mela/_mela.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,052 copying lm_eval/tasks/mela/mela_zh.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,054 copying lm_eval/tasks/mela/mela_ja.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:09,056 creating build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:09,057 copying lm_eval/tasks/ulqa/lambada_uyghur.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:09,060 copying lm_eval/tasks/ulqa/ulqa_.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:09,062 copying lm_eval/tasks/ulqa/celep1.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:09,064 copying lm_eval/tasks/ulqa/celep2.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:09,066 copying lm_eval/tasks/ulqa/uleval.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:09,068 copying lm_eval/tasks/ulqa/ulqa.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:09,070 creating build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:09,071 copying lm_eval/tasks/ulqa/ulut/wsm.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:09,073 copying lm_eval/tasks/ulqa/ulut/nug.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:09,075 copying lm_eval/tasks/ulqa/ulut/wub.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:09,077 copying lm_eval/tasks/ulqa/ulut/ulut.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:09,079 copying lm_eval/tasks/ulqa/ulut/wum.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:09,082 copying lm_eval/tasks/ulqa/ulut/wag.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:09,084 creating build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,085 copying lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,088 copying lm_eval/tasks/pile/pile_stackexchange.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,090 copying lm_eval/tasks/pile/pile_gutenberg.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,092 copying lm_eval/tasks/pile/pile_pile-cc.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,094 copying lm_eval/tasks/pile/pile_europarl.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,096 copying lm_eval/tasks/pile/pile_books3.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,098 copying lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,101 copying lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,103 copying lm_eval/tasks/pile/pile_hackernews.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,105 copying lm_eval/tasks/pile/pile_uspto.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,107 copying lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,109 copying lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,112 copying lm_eval/tasks/pile/pile_github.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,114 copying lm_eval/tasks/pile/pile_philpapers.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,116 copying lm_eval/tasks/pile/pile_wikipedia.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,118 copying lm_eval/tasks/pile/pile_enron.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,120 copying lm_eval/tasks/pile/pile_freelaw.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,123 copying lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,125 copying lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,127 copying lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,129 copying lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,132 copying lm_eval/tasks/pile/pile_arxiv.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:09,134 copying lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,137 copying lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,139 copying lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,141 copying lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,143 copying lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,146 copying lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,148 copying lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,150 copying lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,153 copying lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,156 copying lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,158 copying lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,161 copying lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,163 copying lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,166 copying lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,168 copying lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,171 copying lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,173 copying lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,176 copying lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,178 copying lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,181 copying lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,183 copying lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,185 copying lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,187 copying lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,189 copying lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,192 copying lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,194 copying lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,196 copying lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,198 copying lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,200 copying lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,202 copying lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,205 copying lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,207 copying lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,209 copying lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,211 copying lm_eval/tasks/ceval/_ceval-valid.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,214 copying lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,217 copying lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,219 copying lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,221 copying lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,223 copying lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,226 copying lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,228 copying lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,230 copying lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,231 copying lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,233 copying lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,235 copying lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,237 copying lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,239 copying lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,241 copying lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,244 copying lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,246 copying lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,248 copying lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,250 copying lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,252 copying lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:09,254 creating build/lib/lm_eval/tasks/polemo2 2026-05-11T14:29:09,255 copying lm_eval/tasks/polemo2/polemo2_out.yaml -> build/lib/lm_eval/tasks/polemo2 2026-05-11T14:29:09,257 copying lm_eval/tasks/polemo2/polemo2_in.yaml -> build/lib/lm_eval/tasks/polemo2 2026-05-11T14:29:09,259 copying lm_eval/tasks/coqa/default.yaml -> build/lib/lm_eval/tasks/coqa 2026-05-11T14:29:09,262 copying lm_eval/tasks/olaph/olaph_perplexity.yaml -> build/lib/lm_eval/tasks/olaph 2026-05-11T14:29:09,264 copying lm_eval/tasks/olaph/olaph.yaml -> build/lib/lm_eval/tasks/olaph 2026-05-11T14:29:09,266 copying lm_eval/tasks/click/click.yaml -> build/lib/lm_eval/tasks/click 2026-05-11T14:29:09,268 copying lm_eval/tasks/click/click_lang/click_lang_grammar.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T14:29:09,270 copying lm_eval/tasks/click/click_lang/click_lang_function.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T14:29:09,272 copying lm_eval/tasks/click/click_lang/_click_lang.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T14:29:09,274 copying lm_eval/tasks/click/click_lang/click_lang_text.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T14:29:09,277 copying lm_eval/tasks/click/click_cul/click_cul_history.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,279 copying lm_eval/tasks/click/click_cul/click_cul_kpop.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,281 copying lm_eval/tasks/click/click_cul/_click_cul.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,283 copying lm_eval/tasks/click/click_cul/click_cul_law.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,285 copying lm_eval/tasks/click/click_cul/click_cul_politics.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,287 copying lm_eval/tasks/click/click_cul/click_cul_geography.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,289 copying lm_eval/tasks/click/click_cul/click_cul_economy.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,291 copying lm_eval/tasks/click/click_cul/click_cul_society.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,293 copying lm_eval/tasks/click/click_cul/click_cul_tradition.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:09,296 copying lm_eval/tasks/afrimmlu/direct/afrimmlu.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct 2026-05-11T14:29:09,298 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,300 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,302 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,304 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,306 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,308 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,310 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,312 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,314 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,316 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,318 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,320 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,322 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,325 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,327 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,329 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,331 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,333 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:09,335 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,337 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,340 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,342 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,344 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,346 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,348 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,350 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,353 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,354 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,356 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,358 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,360 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,362 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,364 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,367 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,369 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,371 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:09,374 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,377 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,380 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,383 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,385 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,387 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,390 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,392 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,394 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,396 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,398 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,400 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,403 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,405 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,407 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,410 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,412 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,414 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:09,416 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,418 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,420 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,422 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,424 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,426 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,428 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,430 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,432 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,434 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,436 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,438 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,440 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,442 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,445 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,447 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,449 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,451 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:09,453 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,455 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,457 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,459 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,461 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,463 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,465 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,468 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,470 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,472 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,474 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,476 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,478 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,480 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,482 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,484 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,486 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,488 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:09,490 copying lm_eval/tasks/afrimmlu/translate/afrimmlu_tt.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate 2026-05-11T14:29:09,492 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,494 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,496 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,498 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,500 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,502 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,504 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,507 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,509 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,511 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,513 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,515 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,518 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,520 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,522 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,524 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,526 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:09,528 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,530 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,532 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,534 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,536 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,538 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,540 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,542 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,544 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,547 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,549 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,551 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,553 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,555 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,557 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,559 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,561 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:09,563 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,565 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,568 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,570 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,572 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,574 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,576 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,578 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,580 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,582 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,584 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,586 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,588 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,590 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,592 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,594 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,596 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:09,598 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,600 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,602 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,604 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,606 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,608 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,610 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,612 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,614 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,616 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,618 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,620 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,622 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,624 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,626 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,628 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,631 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:09,633 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,635 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,637 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,638 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,640 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,642 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,644 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,646 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,648 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,650 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,652 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,654 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,656 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,658 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,660 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,662 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,664 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:09,666 creating build/lib/lm_eval/tasks/unscramble 2026-05-11T14:29:09,667 copying lm_eval/tasks/unscramble/random_insertion.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T14:29:09,670 copying lm_eval/tasks/unscramble/anagrams1.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T14:29:09,672 copying lm_eval/tasks/unscramble/reversed_words.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T14:29:09,674 copying lm_eval/tasks/unscramble/anagrams2.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T14:29:09,676 copying lm_eval/tasks/unscramble/cycle_letters.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T14:29:09,678 copying lm_eval/tasks/mbpp/mbpp_plus.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T14:29:09,680 copying lm_eval/tasks/mbpp/mbpp_instruct.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T14:29:09,682 copying lm_eval/tasks/mbpp/mbpp.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T14:29:09,685 copying lm_eval/tasks/mbpp/mbpp_plus_instruct.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T14:29:09,687 creating build/lib/lm_eval/tasks/pile_10k 2026-05-11T14:29:09,688 copying lm_eval/tasks/pile_10k/pile_10k.yaml -> build/lib/lm_eval/tasks/pile_10k 2026-05-11T14:29:09,690 creating build/lib/lm_eval/tasks/nq_open 2026-05-11T14:29:09,691 copying lm_eval/tasks/nq_open/nq_open.yaml -> build/lib/lm_eval/tasks/nq_open 2026-05-11T14:29:09,693 copying lm_eval/tasks/catalan_bench/xnli_va.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,695 copying lm_eval/tasks/catalan_bench/xstorycloze_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,697 copying lm_eval/tasks/catalan_bench/paws_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,699 copying lm_eval/tasks/catalan_bench/cabreu_abstractive.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,701 copying lm_eval/tasks/catalan_bench/xquad_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,703 copying lm_eval/tasks/catalan_bench/catalan_bench.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,705 copying lm_eval/tasks/catalan_bench/wnli_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,707 copying lm_eval/tasks/catalan_bench/siqa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,709 copying lm_eval/tasks/catalan_bench/arc_ca_challenge.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,711 copying lm_eval/tasks/catalan_bench/cocoteros_va.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,713 copying lm_eval/tasks/catalan_bench/coqcat.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,715 copying lm_eval/tasks/catalan_bench/catalanqa.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,717 copying lm_eval/tasks/catalan_bench/cabreu_extreme.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,719 copying lm_eval/tasks/catalan_bench/openbookqa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,721 copying lm_eval/tasks/catalan_bench/copa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,723 copying lm_eval/tasks/catalan_bench/xnli_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,726 copying lm_eval/tasks/catalan_bench/arc_ca_easy.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,728 copying lm_eval/tasks/catalan_bench/teca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,731 copying lm_eval/tasks/catalan_bench/catcola.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,733 copying lm_eval/tasks/catalan_bench/parafraseja.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,735 copying lm_eval/tasks/catalan_bench/piqa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,738 copying lm_eval/tasks/catalan_bench/cabreu_extractive.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,740 copying lm_eval/tasks/catalan_bench/mgsm_direct_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:09,743 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-eu.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,745 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-en.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,747 copying lm_eval/tasks/catalan_bench/flores_ca/flores_eu-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,749 copying lm_eval/tasks/catalan_bench/flores_ca/flores_de-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,752 copying lm_eval/tasks/catalan_bench/flores_ca/flores_es-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,754 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-es.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,756 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-it.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,758 copying lm_eval/tasks/catalan_bench/flores_ca/flores_it-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,760 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-gl.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,762 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-de.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,764 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-pt.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,767 copying lm_eval/tasks/catalan_bench/flores_ca/flores_pt-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,769 copying lm_eval/tasks/catalan_bench/flores_ca/flores_fr-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,771 copying lm_eval/tasks/catalan_bench/flores_ca/flores_en-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,773 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-fr.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,775 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,777 copying lm_eval/tasks/catalan_bench/flores_ca/flores_gl-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:09,780 creating build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:09,781 copying lm_eval/tasks/catalan_bench/phrases_va/phrases_ca-va.yaml -> build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:09,783 copying lm_eval/tasks/catalan_bench/phrases_va/phrases_va-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:09,785 copying lm_eval/tasks/catalan_bench/truthfulqa_va/truthfulqa_va.yaml -> build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T14:29:09,787 copying lm_eval/tasks/bbq/bbq_multiple_choice.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:29:09,789 copying lm_eval/tasks/bbq/bbq_multiple_choice_disambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:29:09,792 copying lm_eval/tasks/bbq/bbq_generate.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:29:09,794 copying lm_eval/tasks/bbq/bbq_generate_disambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:29:09,796 copying lm_eval/tasks/bbq/bbq_generate_ambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:29:09,799 copying lm_eval/tasks/bbq/bbq_multiple_choice_ambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:29:09,801 copying lm_eval/tasks/humaneval_infilling/single_line_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:09,803 copying lm_eval/tasks/humaneval_infilling/humaneval_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:09,805 copying lm_eval/tasks/humaneval_infilling/random_span_infilling_light.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:09,807 copying lm_eval/tasks/humaneval_infilling/multi_line_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:09,809 copying lm_eval/tasks/humaneval_infilling/random_span_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:09,811 creating build/lib/lm_eval/tasks/mc_taco 2026-05-11T14:29:09,812 copying lm_eval/tasks/mc_taco/default.yaml -> build/lib/lm_eval/tasks/mc_taco 2026-05-11T14:29:09,814 creating build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,816 copying lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,818 copying lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,820 copying lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,823 copying lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,825 copying lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,827 copying lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,829 copying lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,832 copying lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,834 copying lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,837 copying lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,839 copying lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:09,841 creating build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,842 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,844 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,846 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,849 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,851 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,853 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,855 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,858 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,860 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,862 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,864 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:09,867 creating build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,868 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,870 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,872 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,874 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,877 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,879 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,881 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,883 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,886 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,888 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,890 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:09,893 copying lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T14:29:09,895 copying lm_eval/tasks/super_glue/multirc/default.yaml -> build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T14:29:09,897 creating build/lib/lm_eval/tasks/super_glue/rte 2026-05-11T14:29:09,898 copying lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/rte 2026-05-11T14:29:09,901 copying lm_eval/tasks/super_glue/rte/default.yaml -> build/lib/lm_eval/tasks/super_glue/rte 2026-05-11T14:29:09,903 copying lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T14:29:09,906 copying lm_eval/tasks/super_glue/wsc/default.yaml -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T14:29:09,908 copying lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T14:29:09,910 copying lm_eval/tasks/super_glue/cb/default.yaml -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T14:29:09,913 copying lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T14:29:09,915 copying lm_eval/tasks/super_glue/copa/default.yaml -> build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T14:29:09,917 creating build/lib/lm_eval/tasks/super_glue/wic 2026-05-11T14:29:09,918 copying lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wic 2026-05-11T14:29:09,920 copying lm_eval/tasks/super_glue/wic/default.yaml -> build/lib/lm_eval/tasks/super_glue/wic 2026-05-11T14:29:09,922 creating build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:09,923 copying lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:09,926 copying lm_eval/tasks/super_glue/boolq/default.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:09,928 copying lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:09,930 copying lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T14:29:09,932 copying lm_eval/tasks/super_glue/record/default.yaml -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T14:29:09,934 copying lm_eval/tasks/medqa/medqa.yaml -> build/lib/lm_eval/tasks/medqa 2026-05-11T14:29:09,937 copying lm_eval/tasks/mimic_repsum/mimic_repsum.yaml -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T14:29:09,939 copying lm_eval/tasks/mimic_repsum/mimic_repsum_perplexity.yaml -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T14:29:09,941 copying lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:09,943 copying lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:09,945 copying lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:09,948 copying lm_eval/tasks/xwinograd/_xwinograd.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:09,950 copying lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:09,952 copying lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:09,954 copying lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:09,956 copying lm_eval/tasks/qasper/freeform.yaml -> build/lib/lm_eval/tasks/qasper 2026-05-11T14:29:09,958 copying lm_eval/tasks/qasper/bool.yaml -> build/lib/lm_eval/tasks/qasper 2026-05-11T14:29:09,961 copying lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,963 copying lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,965 copying lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,967 copying lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,969 copying lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,971 copying lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,973 copying lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,976 copying lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,978 copying lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,980 copying lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,982 copying lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,984 copying lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,986 copying lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,988 copying lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,990 copying lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,992 copying lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,994 copying lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,996 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:09,998 copying lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,001 copying lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,003 copying lm_eval/tasks/tmmluplus/default/_tmmluplus.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,005 copying lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,007 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,009 copying lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,011 copying lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,013 copying lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,015 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,017 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_STEM.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,020 copying lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,022 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,024 copying lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,026 copying lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,028 copying lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,030 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_social_sciences.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,032 copying lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,035 copying lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,037 copying lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,039 copying lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,041 copying lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,043 copying lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,045 copying lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,047 copying lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,049 copying lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,051 copying lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,053 copying lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,055 copying lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,057 copying lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,059 copying lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,062 copying lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,064 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,066 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,068 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_humanities.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,070 copying lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,072 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,074 copying lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,076 copying lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,078 copying lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,081 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_other.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,083 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,085 copying lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,087 copying lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,089 copying lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,091 copying lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,093 copying lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,096 copying lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,098 copying lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,100 copying lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,102 copying lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,104 copying lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,106 copying lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,108 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,110 copying lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:10,112 creating build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,113 copying lm_eval/tasks/turkishmmlu/config/Turkish_Language_and_Literature.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,116 copying lm_eval/tasks/turkishmmlu/config/Biology.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,118 copying lm_eval/tasks/turkishmmlu/config/Religion_and_Ethics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,120 copying lm_eval/tasks/turkishmmlu/config/Chemistry.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,122 copying lm_eval/tasks/turkishmmlu/config/Mathematics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,124 copying lm_eval/tasks/turkishmmlu/config/Physics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,126 copying lm_eval/tasks/turkishmmlu/config/History.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,129 copying lm_eval/tasks/turkishmmlu/config/Philosophy.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,131 copying lm_eval/tasks/turkishmmlu/config/Geography.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:10,133 creating build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,134 copying lm_eval/tasks/turkishmmlu/config_cot/Turkish_Language_and_Literature.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,137 copying lm_eval/tasks/turkishmmlu/config_cot/Biology.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,140 copying lm_eval/tasks/turkishmmlu/config_cot/Religion_and_Ethics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,142 copying lm_eval/tasks/turkishmmlu/config_cot/Chemistry.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,145 copying lm_eval/tasks/turkishmmlu/config_cot/Mathematics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,147 copying lm_eval/tasks/turkishmmlu/config_cot/Physics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,150 copying lm_eval/tasks/turkishmmlu/config_cot/History.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,153 copying lm_eval/tasks/turkishmmlu/config_cot/Philosophy.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,155 copying lm_eval/tasks/turkishmmlu/config_cot/Geography.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:10,158 creating build/lib/lm_eval/tasks/afrimgsm/direct 2026-05-11T14:29:10,159 copying lm_eval/tasks/afrimgsm/direct/afrimgsm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct 2026-05-11T14:29:10,161 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,163 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,165 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,167 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,169 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,171 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,173 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,175 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,177 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,179 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,181 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,183 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,185 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,188 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,190 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,192 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,194 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,197 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,199 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,201 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:10,203 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,204 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,206 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,209 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,211 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,213 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,215 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,218 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,220 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,223 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,225 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,227 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,229 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,232 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,234 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,236 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,238 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,240 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,242 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,244 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:10,246 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,247 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,250 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,252 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,254 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,256 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,258 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,260 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,262 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,264 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,267 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,269 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,271 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,273 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,275 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,278 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,280 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,282 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,284 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,286 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:10,288 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,289 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,292 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,294 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,296 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,298 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,301 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,303 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,306 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,309 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,311 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,314 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,316 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,319 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,322 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,324 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,327 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,329 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,331 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,333 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:10,336 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,337 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,339 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,341 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,344 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,346 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,348 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,350 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,353 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,355 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,357 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,359 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,361 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,364 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,366 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,368 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,370 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,372 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,374 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,376 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:10,378 creating build/lib/lm_eval/tasks/afrimgsm/translate 2026-05-11T14:29:10,379 copying lm_eval/tasks/afrimgsm/translate/afrimgsm_tt.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate 2026-05-11T14:29:10,381 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,383 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,385 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,388 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,390 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,392 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,395 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,397 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,399 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,401 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,404 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,406 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,408 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,410 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,412 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,415 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,417 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,420 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:10,422 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,423 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,425 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,427 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,429 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,431 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,433 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,436 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,438 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,440 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,442 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,445 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,447 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,449 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,451 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,454 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,456 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,458 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:10,460 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,461 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,464 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,466 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,468 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,470 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,473 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,475 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,477 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,479 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,481 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,483 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,485 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,488 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,490 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,492 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,494 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,497 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:10,499 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,500 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,502 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,504 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,507 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,509 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,512 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,515 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,517 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,519 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,522 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,524 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,526 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,529 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,531 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,533 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,535 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,538 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:10,540 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,541 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,543 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,545 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,548 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,550 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,552 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,554 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,556 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,558 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,561 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,563 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,565 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,567 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,570 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,572 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,574 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,576 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:10,578 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T14:29:10,579 copying lm_eval/tasks/afrimgsm/direct_cot/afrimgsm_cot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T14:29:10,581 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,582 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,585 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,587 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,589 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,591 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,593 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,595 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,597 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,600 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,602 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,604 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,607 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,609 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,612 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,614 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,617 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,619 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,621 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,623 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:10,625 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,626 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,628 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,630 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,633 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,635 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,638 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,641 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,643 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,646 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,649 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,652 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,654 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,656 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,659 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,661 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,663 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,665 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,668 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,671 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:10,673 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,675 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,678 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,680 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,683 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,685 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,687 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,689 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,691 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,693 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,695 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,698 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,700 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,702 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,704 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,707 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,709 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,711 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,713 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,715 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:10,717 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,718 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,721 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,723 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,725 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,727 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,729 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,731 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,733 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,735 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,738 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,740 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,742 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,744 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,746 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,749 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,751 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,754 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,757 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,760 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:10,762 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,763 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,766 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,768 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,770 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,773 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,775 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,777 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,779 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,781 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,784 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,786 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,788 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,790 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,792 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,794 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,797 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,799 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,801 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,803 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:10,805 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T14:29:10,807 copying lm_eval/tasks/afrimgsm/translate_cot/afrimgsm_tt_cot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T14:29:10,809 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,810 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,812 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,814 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,817 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,819 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,821 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,823 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,825 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,828 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,830 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,832 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,834 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,836 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,838 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,840 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,843 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,845 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,847 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:10,849 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,850 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,852 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,854 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,857 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,859 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,861 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,863 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,866 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,868 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,870 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,872 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,875 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,877 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,879 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,881 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,883 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,886 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,888 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:10,890 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,891 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,893 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,896 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,898 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,900 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,902 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,905 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,907 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,910 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,912 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,914 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,916 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,918 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,921 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,923 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,925 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,928 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,930 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:10,933 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,934 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,937 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,939 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,942 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,944 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,947 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,949 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,952 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,956 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,958 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,960 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,963 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,965 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,967 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,970 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,972 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,975 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,978 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:10,981 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,982 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,985 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,987 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,989 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,992 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,994 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,997 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:10,999 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,001 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,003 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,005 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,007 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,009 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,012 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,014 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,016 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,018 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,020 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:11,022 creating build/lib/lm_eval/tasks/openbookqa 2026-05-11T14:29:11,024 copying lm_eval/tasks/openbookqa/openbookqa.yaml -> build/lib/lm_eval/tasks/openbookqa 2026-05-11T14:29:11,026 copying lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,028 copying lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,031 copying lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,033 copying lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,035 copying lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,037 copying lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,040 copying lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,042 copying lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,044 copying lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,047 copying lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,049 copying lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,051 copying lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,053 copying lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,056 copying lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,058 copying lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,060 copying lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,062 copying lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,064 copying lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,066 copying lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,068 copying lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,070 copying lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,072 copying lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:11,074 creating build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:11,075 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/_sciknoweval_mcqa.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:11,077 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Material.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:11,079 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Physics.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:11,082 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Biology.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:11,084 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Chemistry.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:11,086 copying lm_eval/tasks/e2lmc/mmlu_early_training/mmlu_early_training.yaml -> build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:29:11,088 copying lm_eval/tasks/e2lmc/noor/noor_high_school_statistics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,090 copying lm_eval/tasks/e2lmc/noor/noor_college_physics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,092 copying lm_eval/tasks/e2lmc/noor/noor_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,095 copying lm_eval/tasks/e2lmc/noor/_noor_stem.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,097 copying lm_eval/tasks/e2lmc/noor/_noor.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,099 copying lm_eval/tasks/e2lmc/noor/noor_conceptual_physics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,101 copying lm_eval/tasks/e2lmc/noor/noor_machine_learning.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,104 copying lm_eval/tasks/e2lmc/noor/noor_abstract_algebra.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,106 copying lm_eval/tasks/e2lmc/noor/noor_college_mathematics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,108 copying lm_eval/tasks/e2lmc/noor/noor_high_school_physics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,110 copying lm_eval/tasks/e2lmc/noor/noor_college_computer_science.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,112 copying lm_eval/tasks/e2lmc/noor/noor_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,114 copying lm_eval/tasks/e2lmc/noor/noor_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,117 copying lm_eval/tasks/e2lmc/noor/noor_electrical_engineering.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:11,119 creating build/lib/lm_eval/tasks/headqa 2026-05-11T14:29:11,120 copying lm_eval/tasks/headqa/headqa_es.yaml -> build/lib/lm_eval/tasks/headqa 2026-05-11T14:29:11,122 copying lm_eval/tasks/headqa/headqa_en.yaml -> build/lib/lm_eval/tasks/headqa 2026-05-11T14:29:11,124 copying lm_eval/tasks/libra/librusec_history.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,127 copying lm_eval/tasks/libra/ru_babilong_qa5.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,129 copying lm_eval/tasks/libra/matreshka_names.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,131 copying lm_eval/tasks/libra/ru_quality.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,133 copying lm_eval/tasks/libra/ru_2wikimultihopqa.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,135 copying lm_eval/tasks/libra/matreshka_yes_no.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,137 copying lm_eval/tasks/libra/_complex_reasoning_and_mathematical_problems.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,139 copying lm_eval/tasks/libra/ru_babilong_qa1.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,141 copying lm_eval/tasks/libra/passkey.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,144 copying lm_eval/tasks/libra/ru_qasper.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,146 copying lm_eval/tasks/libra/ru_babilong_qa4.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,148 copying lm_eval/tasks/libra/_question_answering_and_multiple_choice.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,150 copying lm_eval/tasks/libra/ru_babilong_qa3.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,152 copying lm_eval/tasks/libra/librusec_mhqa.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,154 copying lm_eval/tasks/libra/ru_sci_abstract_retrieval.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,156 copying lm_eval/tasks/libra/long_context_multiq.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,159 copying lm_eval/tasks/libra/ru_gsm100.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,161 copying lm_eval/tasks/libra/passkey_with_librusec.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,163 copying lm_eval/tasks/libra/ru_babilong_qa2.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,165 copying lm_eval/tasks/libra/_simple_information_retrieval.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,167 copying lm_eval/tasks/libra/ru_sci_passage_count.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,169 copying lm_eval/tasks/libra/_multi_hop_question_answering.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:11,172 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light 2026-05-11T14:29:11,174 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_exams_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:29:11,177 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_leaderboard_arabic_exams_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:29:11,179 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,181 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,183 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,185 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_msa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,188 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_dialects_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,190 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_sentiment_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,192 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,194 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_mcq_exams_test_ar_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,196 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,198 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:11,200 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_mt_sciq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:29:11,202 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_leaderboard_arabic_mt_sciq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:29:11,205 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_leaderboard_arabic_mt_boolq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:29:11,207 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_mt_boolq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:29:11,209 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_leaderboard_arabic_mt_piqa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:29:11,211 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_mt_piqa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:29:11,214 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_mt_openbook_qa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:29:11,216 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_leaderboard_arabic_mt_openbook_qa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:29:11,218 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_scenarios_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,220 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_sociology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,222 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_computer_science_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,224 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_virology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,226 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_us_foreign_policy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,229 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,231 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_sexuality_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,233 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_computer_science_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,235 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_abstract_algebra_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,237 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_international_law_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,239 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_physics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,241 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_management_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,244 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_chemistry_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,246 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_geography_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,248 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_medical_genetics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,250 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_nutrition_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,252 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_electrical_engineering_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,254 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_biology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,257 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_law_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,259 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_logical_fallacies_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,261 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_world_religions_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,263 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_psychology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,265 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_mathematics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,267 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_disputes_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,269 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_miscellaneous_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,271 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_business_ethics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,273 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_anatomy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,275 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_psychology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,278 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,280 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_accounting_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,282 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_philosophy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,285 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_world_history_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,287 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_global_facts_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,289 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_prehistory_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,291 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_marketing_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,294 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_astronomy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,296 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_aging_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,298 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_european_history_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,300 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_formal_logic_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,302 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_public_relations_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,304 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_security_studies_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,306 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_medicine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,309 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_machine_learning_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,311 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,313 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_statistics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,315 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_clinical_knowledge_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,317 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_biology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,320 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_mathematics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,322 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_conceptual_physics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,324 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_microeconomics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,327 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_physics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,329 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_jurisprudence_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,331 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_us_history_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,333 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_chemistry_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,335 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_elementary_mathematics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,338 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_econometrics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,340 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_medicine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,342 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_computer_security_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:11,344 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arbic_leaderboard_arabic_mt_copa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:29:11,346 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arabic_mt_copa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:29:11,349 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_leaderboard_arabic_mt_arc_easy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:29:11,351 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_mt_arc_easy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:29:11,353 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_mt_mmlu_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:29:11,355 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_leaderboard_arabic_mt_mmlu_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:29:11,357 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_leaderboard_arabic_mt_toxigen_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:29:11,360 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_mt_toxigen_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:29:11,362 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_mt_arc_challenge_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:29:11,364 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_leaderboard_arabic_mt_arc_challenge_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:29:11,366 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Syria_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,368 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_daily_life_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,370 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Astronomy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,372 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arab_Empire_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,375 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Somalia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,377 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Language_Origin_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,378 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromChina_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,380 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Art_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,383 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromAncientEgypt_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,385 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Math_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,387 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromRome_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,389 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_branches_and_schools_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,391 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Lebanon_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,393 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Culture_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,396 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ornament_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,398 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Oman_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,400 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Philosophy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,402 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islamic_law_system_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,404 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Algeria_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,406 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Qatar_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,408 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Kuwait_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,410 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Clothing_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,412 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Ancient_Egypt_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,415 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_History_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,417 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,419 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mauritania_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,421 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Funeral_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,423 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ceremony_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,426 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Architecture_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,428 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Egypt_modern_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,430 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_communication_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,432 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Palestine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,434 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromIslam_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,437 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Wedding_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,439 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_computer_and_phone_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,441 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Tunisia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,443 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_United_Arab_Emirates_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,446 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Geography_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,448 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromPersia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,450 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mesopotamia_civilization_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,452 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Calligraphy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,454 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_Education_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,456 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Music_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,459 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Saudi_Arabia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,461 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromByzantium_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,463 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Medicine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,465 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromGreece_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,468 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Sudan_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,470 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Comoros_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,472 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Bahrain_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,475 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Morocco_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,477 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,479 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Libya_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,481 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Iraq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,484 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Yemen_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,486 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Literature_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,488 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Food_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,491 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_entertainment_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,493 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Jordan_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:11,495 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_leaderboard_arabic_mt_race_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:29:11,497 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_mt_race_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:29:11,500 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_leaderboard_arabic_mt_hellaswag_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:29:11,502 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_mt_hellaswag_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:29:11,504 copying lm_eval/tasks/slr_bench/slr_bench_medium.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:11,506 copying lm_eval/tasks/slr_bench/slr_bench_group.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:11,508 copying lm_eval/tasks/slr_bench/slr_bench_easy.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:11,511 copying lm_eval/tasks/slr_bench/slr_bench_basic.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:11,513 copying lm_eval/tasks/slr_bench/slr_bench_hard.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:11,515 copying lm_eval/tasks/slr_bench/slr_bench_all.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:11,517 copying lm_eval/tasks/logiqa2/logieval.yaml -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T14:29:11,520 copying lm_eval/tasks/logiqa2/logiqa2.yaml -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T14:29:11,522 copying lm_eval/tasks/longbench/repobench-p.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,525 copying lm_eval/tasks/longbench/2wikimqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,527 copying lm_eval/tasks/longbench/_longbench_multi_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,529 copying lm_eval/tasks/longbench/musique.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,531 copying lm_eval/tasks/longbench/_longbench.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,534 copying lm_eval/tasks/longbench/_longbench_single.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,536 copying lm_eval/tasks/longbench/multifieldqa_zh.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,538 copying lm_eval/tasks/longbench/lcc.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,540 copying lm_eval/tasks/longbench/_longbench_code.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,543 copying lm_eval/tasks/longbench/gov_report_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,545 copying lm_eval/tasks/longbench/triviaqa_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,548 copying lm_eval/tasks/longbench/passage_retrieval_zh.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,550 copying lm_eval/tasks/longbench/qasper.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,552 copying lm_eval/tasks/longbench/qmsum.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,554 copying lm_eval/tasks/longbench/_longbench_synthetic.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,556 copying lm_eval/tasks/longbench/2wikimqa_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,558 copying lm_eval/tasks/longbench/multifieldqa_en.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,560 copying lm_eval/tasks/longbench/multi_news_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,562 copying lm_eval/tasks/longbench/_longbench_fewshot.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,564 copying lm_eval/tasks/longbench/_longbench_single_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,566 copying lm_eval/tasks/longbench/triviaqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,569 copying lm_eval/tasks/longbench/_longbench_fewshot_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,571 copying lm_eval/tasks/longbench/hotpotqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,573 copying lm_eval/tasks/longbench/trec.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,575 copying lm_eval/tasks/longbench/passage_retrieval_en.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,577 copying lm_eval/tasks/longbench/passage_retrieval_en_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,580 copying lm_eval/tasks/longbench/passage_count_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,582 copying lm_eval/tasks/longbench/_longbench_code_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,584 copying lm_eval/tasks/longbench/repobench-p_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,586 copying lm_eval/tasks/longbench/lcc_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,589 copying lm_eval/tasks/longbench/multifieldqa_en_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,591 copying lm_eval/tasks/longbench/samsum.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,594 copying lm_eval/tasks/longbench/narrativeqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,597 copying lm_eval/tasks/longbench/_longbench_summarization.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,599 copying lm_eval/tasks/longbench/gov_report.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,601 copying lm_eval/tasks/longbench/qasper_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,604 copying lm_eval/tasks/longbench/lsht.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,606 copying lm_eval/tasks/longbench/dureader.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,609 copying lm_eval/tasks/longbench/vcsum.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,611 copying lm_eval/tasks/longbench/_longbench_summarization_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,613 copying lm_eval/tasks/longbench/_longbench_multi.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,615 copying lm_eval/tasks/longbench/trec_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,617 copying lm_eval/tasks/longbench/multi_news.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,619 copying lm_eval/tasks/longbench/_longbench_synthetic_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,621 copying lm_eval/tasks/longbench/passage_count.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,624 copying lm_eval/tasks/longbench/_longbench_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,626 copying lm_eval/tasks/longbench/samsum_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,628 copying lm_eval/tasks/longbench/hotpotqa_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:11,631 creating build/lib/lm_eval/tasks/asdiv 2026-05-11T14:29:11,632 copying lm_eval/tasks/asdiv/asdiv-cot-llama.yaml -> build/lib/lm_eval/tasks/asdiv 2026-05-11T14:29:11,634 copying lm_eval/tasks/asdiv/default.yaml -> build/lib/lm_eval/tasks/asdiv 2026-05-11T14:29:11,636 copying lm_eval/tasks/basqueglue/coref.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:29:11,638 copying lm_eval/tasks/basqueglue/qnli.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:29:11,641 copying lm_eval/tasks/basqueglue/bec.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:29:11,643 copying lm_eval/tasks/basqueglue/vaxx.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:29:11,645 copying lm_eval/tasks/basqueglue/bhtc.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:29:11,647 copying lm_eval/tasks/basqueglue/wic.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:29:11,650 copying lm_eval/tasks/cruxeval/cruxeval_output_cot.yaml -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:11,652 copying lm_eval/tasks/cruxeval/cruxeval_output.yaml -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:11,654 copying lm_eval/tasks/cruxeval/cruxeval_input_08.yaml -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:11,656 copying lm_eval/tasks/cruxeval/cruxeval_input.yaml -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:11,659 copying lm_eval/tasks/cruxeval/cruxeval_input_cot.yaml -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:11,661 copying lm_eval/tasks/cruxeval/cruxeval_output_08.yaml -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:11,663 copying lm_eval/tasks/mathqa/mathqa.yaml -> build/lib/lm_eval/tasks/mathqa 2026-05-11T14:29:11,665 creating build/lib/lm_eval/tasks/alghafa/copa_ar 2026-05-11T14:29:11,667 copying lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml -> build/lib/lm_eval/tasks/alghafa/copa_ar 2026-05-11T14:29:11,669 creating build/lib/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T14:29:11,670 copying lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml -> build/lib/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T14:29:11,672 creating build/lib/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T14:29:11,674 copying lm_eval/tasks/mmlu-redux-spanish/mmlu-redux-2.0-spanish.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T14:29:11,676 creating build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,677 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,680 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,682 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,684 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,686 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,688 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,690 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,692 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,694 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,697 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,699 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,701 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,703 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,705 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,707 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,709 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,711 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,713 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,716 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,718 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,720 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,722 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,724 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,727 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,729 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,731 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,733 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,735 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,738 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,740 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,742 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,744 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,746 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,748 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,750 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,752 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,754 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,756 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,758 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,760 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,762 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,764 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,766 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,768 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,770 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,772 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,774 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,777 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,779 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,781 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,783 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,785 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,787 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,790 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,792 copying lm_eval/tasks/mmlu-redux-spanish/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,794 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,796 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,798 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:11,800 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_mac.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:11,802 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_myc.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:11,804 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:11,807 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msda.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:11,809 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_electrom.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:11,811 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msac.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:11,813 copying lm_eval/tasks/darija_bench/darija_summarization/summarization.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:11,815 copying lm_eval/tasks/darija_bench/darija_summarization/summarization_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:11,817 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_en_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,820 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,822 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_fr_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,824 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,826 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_fr_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,828 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_en.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,830 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,832 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,835 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_fr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,837 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_fr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,839 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_dr_msa.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,842 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_msa_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,844 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_en.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,846 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_msa_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,848 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,850 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_msa.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,852 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_msa_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,854 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_en_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,856 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_msa.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,859 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,861 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_dr_en.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,863 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_en_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,865 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,867 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,869 copying lm_eval/tasks/darija_bench/darija_translation/translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:11,871 copying lm_eval/tasks/darija_bench/darija_transliteration/translation_dr_ar.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:11,874 copying lm_eval/tasks/darija_bench/darija_transliteration/transliteration_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:11,876 copying lm_eval/tasks/darija_bench/darija_transliteration/translation_ar_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:11,878 copying lm_eval/tasks/darija_bench/darija_transliteration/transliteration_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:11,880 creating build/lib/lm_eval/tasks/siqa 2026-05-11T14:29:11,881 copying lm_eval/tasks/siqa/siqa.yaml -> build/lib/lm_eval/tasks/siqa 2026-05-11T14:29:11,883 copying lm_eval/tasks/hellaswag/hellaswag.yaml -> build/lib/lm_eval/tasks/hellaswag 2026-05-11T14:29:11,885 creating build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,886 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,889 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,891 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,893 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,895 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,897 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,900 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,902 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,904 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,906 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,908 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,910 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__plur_MS_ANPHR.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,912 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,914 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,916 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,919 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,921 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__sing_MS_MV_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,923 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,925 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,928 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,930 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,932 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,934 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,936 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,939 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,941 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,943 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,945 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,948 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,950 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,952 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,954 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,956 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,958 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,960 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,962 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__plur_MS_MV_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,964 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,967 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__sing_MS_ANPHR.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,969 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,971 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,973 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,975 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,977 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,979 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,981 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,983 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,986 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,988 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,990 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,992 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,994 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,996 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:11,998 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,000 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,003 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__plur_MS_LMV_LMV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,005 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,007 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__sing_MS_LMV_LMV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,009 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,011 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,014 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,016 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,018 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,020 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,022 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,024 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,026 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,029 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,031 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,033 copying lm_eval/tasks/lm_syneval/lm_syneval_group.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,036 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,038 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,040 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,042 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:12,044 creating build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,045 copying lm_eval/tasks/blimp_nl/adverbial_modification__position_type.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,048 copying lm_eval/tasks/blimp_nl/extraposition__argument_nominal.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,050 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,052 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,054 copying lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,057 copying lm_eval/tasks/blimp_nl/passive__impersonal.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,059 copying lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,061 copying lm_eval/tasks/blimp_nl/topicalization__island.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,063 copying lm_eval/tasks/blimp_nl/relativization__island.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,066 copying lm_eval/tasks/blimp_nl/determiners__geen_expletive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,068 copying lm_eval/tasks/blimp_nl/binding_principle_a__c_command.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,070 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,072 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,074 copying lm_eval/tasks/blimp_nl/argument_structure__argument_number_in_transitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,076 copying lm_eval/tasks/blimp_nl/complementive__ditransitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,078 copying lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,080 copying lm_eval/tasks/blimp_nl/finite_argument_clause__perception_dat.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,082 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,085 copying lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,087 copying lm_eval/tasks/blimp_nl/extraposition__adjectival_supplementive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,089 copying lm_eval/tasks/blimp_nl/extraposition__adjectival_adverbial.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,091 copying lm_eval/tasks/blimp_nl/binding_principle_a__monomorphemic.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,093 copying lm_eval/tasks/blimp_nl/wh_movement__stranding_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,096 copying lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,098 copying lm_eval/tasks/blimp_nl/auxiliaries__order_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,101 copying lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,103 copying lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,105 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,108 copying lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,111 copying lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,113 copying lm_eval/tasks/blimp_nl/adpositional_phrases__argument_scrambling.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,116 copying lm_eval/tasks/blimp_nl/verb_second__order_main.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,118 copying lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,121 copying lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,123 copying lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,125 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__resumptive_prolepsis.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,128 copying lm_eval/tasks/blimp_nl/crossing_dependencies__cross_dependency.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,130 copying lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,132 copying lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,135 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,138 copying lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,140 copying lm_eval/tasks/blimp_nl/blimp_nl_group.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,143 copying lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,145 copying lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,147 copying lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,149 copying lm_eval/tasks/blimp_nl/complementive__position_adverb.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,152 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,154 copying lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,156 copying lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,158 copying lm_eval/tasks/blimp_nl/anaphor_agreement__person.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,160 copying lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,162 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__superiority.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,164 copying lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,167 copying lm_eval/tasks/blimp_nl/adverbial_modification__position_proform.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,169 copying lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,171 copying lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,173 copying lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,175 copying lm_eval/tasks/blimp_nl/adpositional_phrases__argument_r_extraction.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,178 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,180 copying lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,182 copying lm_eval/tasks/blimp_nl/passive__aci.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,185 copying lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,188 copying lm_eval/tasks/blimp_nl/r_words__adverbial.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,192 copying lm_eval/tasks/blimp_nl/complementive__intransitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,196 copying lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,199 copying lm_eval/tasks/blimp_nl/determiners__geen_scrambling_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,201 copying lm_eval/tasks/blimp_nl/complementive__position_verb.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,204 copying lm_eval/tasks/blimp_nl/argument_structure__argument_number_ditransitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,207 copying lm_eval/tasks/blimp_nl/auxiliaries__perfect.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,209 copying lm_eval/tasks/blimp_nl/wh_movement__stranding_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,212 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,214 copying lm_eval/tasks/blimp_nl/anaphor_agreement__number.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,217 copying lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,219 copying lm_eval/tasks/blimp_nl/auxiliaries__order_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,222 copying lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,224 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,227 copying lm_eval/tasks/blimp_nl/determiners__negative_polarity.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,229 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,232 copying lm_eval/tasks/blimp_nl/determiners__geen_scrambling_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,235 copying lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,237 copying lm_eval/tasks/blimp_nl/finite_argument_clause__complementizer.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,239 copying lm_eval/tasks/blimp_nl/complementive__transitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,242 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,244 copying lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,247 copying lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:12,249 copying lm_eval/tasks/afrobench/afrobench.yaml -> build/lib/lm_eval/tasks/afrobench 2026-05-11T14:29:12,252 copying lm_eval/tasks/afrobench/afrobench-lite.yaml -> build/lib/lm_eval/tasks/afrobench 2026-05-11T14:29:12,254 copying lm_eval/tasks/afrobench/naijarc/naijarc.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T14:29:12,256 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:12,258 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:12,260 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:12,263 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:12,265 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:12,266 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:12,269 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:12,272 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:12,274 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:12,275 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:12,278 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:12,280 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:12,283 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:12,284 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:12,287 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:12,290 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:12,292 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:12,294 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:12,296 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:12,299 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:12,301 copying lm_eval/tasks/afrobench/ntrex/ntrex.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T14:29:12,303 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,305 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,308 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,311 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,314 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,317 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,320 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,323 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,325 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,328 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,330 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,332 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,335 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,338 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,340 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,342 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,345 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,347 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,350 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,352 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,355 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,357 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,360 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,362 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,365 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,367 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,370 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,372 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,375 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,378 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,380 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,383 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,385 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:12,388 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,389 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,392 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,394 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,397 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,399 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,402 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,405 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,407 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,410 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,412 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,415 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,417 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,420 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,422 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,425 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,427 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,430 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,432 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,434 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,437 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,439 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,442 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,444 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,447 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,449 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,452 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,455 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,458 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,461 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,464 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,469 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,471 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:12,473 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,475 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,477 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,479 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,481 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,483 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,485 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,488 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,490 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,492 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,495 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,499 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,503 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,505 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,507 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,509 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,511 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,513 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,516 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,518 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,520 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,527 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,529 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,531 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,533 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,536 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,539 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,541 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,543 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,545 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,547 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,550 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,552 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:12,554 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,555 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,557 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,559 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,562 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,564 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,566 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,569 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,572 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,574 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,576 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,578 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,580 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,582 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,584 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,586 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,588 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,590 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,592 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,594 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,597 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,599 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,601 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,603 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,606 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,608 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,610 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,612 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,614 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,617 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,619 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,621 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,623 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:12,625 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,627 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,629 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,631 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,633 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,636 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,638 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,640 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,642 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,644 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,646 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,649 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,651 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,653 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,655 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,658 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,661 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,663 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,666 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,669 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,671 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,674 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,677 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,679 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,682 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,684 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,686 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,688 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,690 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,693 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,695 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,698 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,701 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:12,703 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,705 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,708 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,711 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,715 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,719 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,721 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,724 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,727 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,730 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,733 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,735 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,738 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,741 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,743 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,746 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,749 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,752 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,755 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,758 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,761 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,764 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,766 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,769 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,772 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,775 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,778 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,781 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,784 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,787 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,789 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,792 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,795 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:12,798 copying lm_eval/tasks/afrobench/afrisenti/afrisenti.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:12,800 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,802 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,804 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,806 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,808 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,810 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,812 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,815 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,817 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,819 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,822 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,824 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,826 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,828 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:12,831 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,833 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,835 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,837 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,839 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,842 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,844 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,846 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,848 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,850 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,853 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,855 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,857 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,860 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:12,862 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,864 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,866 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,868 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,870 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,873 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,875 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,877 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,879 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,881 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,883 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,885 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,888 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,890 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:12,892 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,895 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,897 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,899 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,901 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,904 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,906 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,908 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,910 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,912 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,915 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,917 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,919 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,921 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:12,923 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,925 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,927 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,929 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,931 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,934 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,936 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,939 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,941 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,943 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,945 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,947 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,949 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,951 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:12,954 copying lm_eval/tasks/afrobench/xlsum/xlsum.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T14:29:12,956 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_arabic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,959 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_pidgin.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,961 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_igbo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,963 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_oromo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,965 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_hausa.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,967 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_kirundi.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,970 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_swahili.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,972 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_tigrinya.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,974 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_somali.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,976 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_telugu.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,979 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_yoruba.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,981 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_amharic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:12,983 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_arabic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:12,985 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_pidgin.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:12,987 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_igbo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:12,989 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_oromo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:12,991 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_hausa.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:12,993 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_kirundi.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:12,995 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_swahili.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:12,998 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_tigrinya.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:13,000 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_somali.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:13,002 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_telugu.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:13,004 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_yoruba.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:13,006 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_amharic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:13,009 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_arabic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,011 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_pidgin.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,013 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_igbo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,015 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_oromo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,017 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_hausa.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,019 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_kirundi.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,021 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_swahili.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,023 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_tigrinya.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,025 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_somali.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,028 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_telugu.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,030 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_yoruba.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,032 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_amharic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:13,034 copying lm_eval/tasks/afrobench/openai_mmlu/openai_mmlu.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:29:13,036 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:13,037 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:13,039 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:13,041 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:13,043 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:13,044 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:13,046 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:13,048 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:13,050 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:13,051 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:13,053 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:13,056 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:13,057 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:13,058 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:13,061 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:13,063 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:13,065 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:13,066 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:13,068 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:13,070 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:13,072 copying lm_eval/tasks/afrobench/uhura-arc-easy/uhura.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:29:13,074 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:13,077 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:13,079 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:13,081 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:13,083 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:13,085 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:13,088 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:13,090 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:13,092 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:13,094 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:13,096 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:13,098 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:13,100 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:13,102 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:13,105 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:13,107 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:13,109 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:13,111 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:13,113 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:13,116 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:13,118 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:13,120 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:13,123 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:13,125 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:13,127 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:13,129 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:13,132 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:13,134 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:13,136 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:13,138 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:13,141 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:13,143 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:13,145 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:13,147 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:13,149 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:13,152 copying lm_eval/tasks/afrobench/masakhapos/masakhapos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:29:13,154 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,156 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,158 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,160 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,163 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,165 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,167 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,169 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,172 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,174 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,176 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,178 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,181 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,183 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,185 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,187 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,190 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,192 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,194 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,196 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:13,199 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,201 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,203 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,205 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,208 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,210 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,212 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,215 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,217 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,219 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,221 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,224 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,226 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,229 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,232 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,234 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,237 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,240 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,242 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,245 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:13,247 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,250 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,252 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,254 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,257 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,259 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,261 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,264 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,266 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,268 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,270 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,272 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,275 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,277 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,279 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,281 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,283 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,285 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,287 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,290 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:13,292 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,294 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,296 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,299 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,301 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,303 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,305 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,307 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,309 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,312 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,314 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,316 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,318 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,320 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,323 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,325 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,327 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,329 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,331 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,333 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:13,335 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,337 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,339 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,341 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,344 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,346 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,348 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,350 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,352 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,354 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,356 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,358 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,360 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,362 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,365 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,367 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,369 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,371 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,373 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,375 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:13,378 copying lm_eval/tasks/afrobench/flores/flores.yaml -> build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T14:29:13,380 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,381 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-swh_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,384 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,386 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-luo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,388 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kea_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,390 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,392 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lua_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,394 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-twi_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,396 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,398 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,400 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,402 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,405 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,407 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kab_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,409 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,411 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,413 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,415 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,418 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,420 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-mos_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,422 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sot_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,424 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,426 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-umb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,429 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,431 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,433 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,435 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,437 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,439 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sag_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,441 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,443 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,445 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aka_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,447 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,449 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,451 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lug_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,454 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,456 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,458 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,460 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,463 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-run_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,465 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,467 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ary_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,469 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,471 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sun_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,474 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-plt_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,476 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,478 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,480 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-arz_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,482 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,484 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,486 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nus_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,488 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ban_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,491 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,493 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,495 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,497 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,499 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,501 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,504 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tum_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,506 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,508 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,510 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-acq_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,512 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:13,514 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,515 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,517 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,520 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,522 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tum_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,524 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,526 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,528 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aka_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,530 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,532 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,534 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,537 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,539 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,541 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kab_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,543 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sun_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,545 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ban_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,548 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,550 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,552 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,554 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_mos_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,557 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_plt_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,559 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,561 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,563 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,565 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,568 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,570 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sag_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,572 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,574 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lug_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,576 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_acq_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,579 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,581 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lua_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,583 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,585 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,587 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,589 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,592 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,594 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_arz_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,596 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_run_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,598 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_swh_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,601 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,603 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,605 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kea_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,608 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,610 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,612 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_twi_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,615 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,617 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,619 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,621 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,623 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,626 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,628 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_luo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,630 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,632 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,634 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sot_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,637 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,639 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ary_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,642 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,644 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,646 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,648 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_umb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,651 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nus_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:13,653 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,654 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-swh_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,657 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,659 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-luo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,661 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kea_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,663 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,666 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lua_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,668 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-twi_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,670 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,672 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,674 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,676 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,680 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,682 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kab_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,684 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,686 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,689 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,691 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,693 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,695 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-mos_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,698 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sot_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,700 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,702 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-umb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,705 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,707 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,709 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,712 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,714 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,717 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sag_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,719 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,721 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,723 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aka_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,726 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,728 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,730 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lug_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,732 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,734 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,736 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,739 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,741 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-run_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,743 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,746 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ary_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,748 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,750 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sun_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,752 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-plt_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,755 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,757 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,759 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-arz_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,761 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,764 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,766 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nus_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,768 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ban_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,770 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,772 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,774 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,777 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,779 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,781 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,783 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tum_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,786 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,788 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,790 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-acq_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,793 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:13,795 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,796 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,798 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,801 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,803 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tum_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,805 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,807 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,810 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aka_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,813 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,815 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,818 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,820 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,823 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,825 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kab_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,828 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sun_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,830 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ban_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,832 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,835 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,837 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,839 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_mos_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,842 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_plt_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,844 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,846 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,849 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,851 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,854 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,856 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sag_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,858 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,861 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lug_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,863 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_acq_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,865 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,868 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lua_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,870 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,872 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,874 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,876 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,878 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,880 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_arz_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,882 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_run_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,885 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_swh_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,887 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,889 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,891 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kea_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,893 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,896 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,898 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_twi_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,900 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,902 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,904 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,907 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,909 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,912 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,915 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_luo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,917 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,920 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,923 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sot_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,925 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,927 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ary_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,929 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,931 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,933 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,935 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_umb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,937 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nus_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:13,939 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,941 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-swh_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,943 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,945 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-luo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,948 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kea_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,950 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,952 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lua_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,954 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-twi_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,956 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,959 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,961 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,963 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,965 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,967 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kab_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,969 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,971 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,973 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,975 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,978 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,980 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-mos_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,983 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sot_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,986 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,989 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-umb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,991 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,993 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,995 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:13,998 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,000 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,002 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sag_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,004 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,007 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,009 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aka_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,011 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,014 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,016 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lug_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,018 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,020 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,022 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,025 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,027 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-run_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,029 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,031 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ary_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,034 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,036 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sun_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,038 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-plt_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,041 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,043 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,045 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-arz_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,048 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,051 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,053 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nus_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,056 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ban_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,058 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,060 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,062 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,065 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,067 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,070 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,072 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tum_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,074 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,076 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,079 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-acq_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,081 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:14,083 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,084 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,086 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,088 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,091 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tum_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,093 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,095 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,098 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aka_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,100 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,102 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,105 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,107 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,110 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,112 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kab_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,114 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sun_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,117 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ban_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,119 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,121 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,123 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,125 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_mos_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,127 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_plt_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,129 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,132 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,134 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,136 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,138 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,141 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sag_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,144 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,148 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lug_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,151 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_acq_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,154 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,156 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lua_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,158 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,160 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,163 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,165 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,167 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,169 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_arz_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,171 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_run_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,173 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_swh_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,175 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,177 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,179 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kea_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,182 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,184 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,187 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_twi_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,189 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,192 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,194 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,197 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,200 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,202 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,205 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_luo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,207 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,209 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,211 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sot_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,214 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,217 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ary_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,220 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,222 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,224 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,226 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_umb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,228 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nus_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:14,231 copying lm_eval/tasks/afrobench/adr/afridiacritics.yaml -> build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T14:29:14,232 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:14,233 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:14,236 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:14,237 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:14,239 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:14,241 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:14,243 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:14,244 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:14,246 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:14,249 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:14,251 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:14,253 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:14,255 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:14,256 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:14,258 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:14,260 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:14,263 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:14,265 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:14,267 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:14,269 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:14,271 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:14,274 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:14,276 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:14,278 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:14,280 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:14,281 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:14,283 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:14,285 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:14,287 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:14,289 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:14,291 copying lm_eval/tasks/afrobench/belebele/belebele.yaml -> build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T14:29:14,293 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,294 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,296 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,299 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,301 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,303 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,305 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,307 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,310 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,312 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,315 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,317 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,319 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,322 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,324 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,326 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,328 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,330 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,333 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,335 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,337 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,339 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,341 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,343 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,345 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,347 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,349 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,352 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,354 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,356 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,358 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,360 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:14,363 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,364 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,366 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,369 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,371 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,373 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,375 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,377 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,380 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,382 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,385 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,387 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,390 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,393 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,395 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,398 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,400 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,403 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,405 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,408 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,410 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,412 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,414 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,416 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,418 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,420 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,422 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,424 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,426 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,428 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,431 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,433 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:14,435 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,436 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,438 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,440 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,442 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,445 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,447 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,449 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,451 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,453 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,456 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,458 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,460 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,462 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,465 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,467 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,469 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,472 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,474 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,476 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,478 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,480 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,483 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,485 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,487 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,489 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,492 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,494 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,496 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,498 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,500 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,503 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:14,505 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,506 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,509 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,512 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,514 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,516 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,519 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,521 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,523 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,526 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,528 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,530 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,532 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,534 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,536 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,538 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,540 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,542 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,544 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,547 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,549 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,551 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,553 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,555 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,557 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,560 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,562 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,564 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,566 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,569 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,571 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,573 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:14,575 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,577 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,579 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,581 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,583 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,585 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,587 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,590 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,592 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,594 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,596 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,598 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,601 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,603 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,605 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,607 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,610 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,612 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,614 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,617 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,619 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,621 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,623 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,625 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,627 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,629 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,632 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,634 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,636 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,639 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,641 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,643 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:14,645 copying lm_eval/tasks/afrobench/mafand/mafand.yaml -> build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T14:29:14,648 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-nya.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,650 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-mos.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,652 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bam.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,654 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-swa.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,656 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-sna.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,658 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-ewe.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,660 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-wol.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,662 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-pcm.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,664 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-luo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,666 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-kin.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,668 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-hau.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,670 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,672 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-tsn.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,675 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bbj.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,677 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-amh.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,679 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-yor.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,681 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-xho.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,683 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-fon.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,685 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-lug.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,688 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-zul.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,690 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-twi.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:14,692 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_pcm-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,694 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ewe-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,696 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_swa-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,698 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_luo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,700 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_xho-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,702 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_twi-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,704 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_sna-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,706 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bam-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,708 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bbj-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,711 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_lug-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,713 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_hau-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,715 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_yor-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,717 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_nya-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,719 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_kin-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,721 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_tsn-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,723 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_amh-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,725 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_wol-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,728 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_mos-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,730 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_zul-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,732 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ibo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,734 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_fon-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:14,737 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-nya.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,739 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-mos.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,741 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bam.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,743 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-swa.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,745 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-sna.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,747 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-ewe.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,749 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-wol.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,751 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-pcm.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,753 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-luo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,755 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-kin.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,758 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-hau.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,760 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,762 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-tsn.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,764 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bbj.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,766 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-amh.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,768 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-yor.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,771 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-xho.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,773 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-fon.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,775 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-lug.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,777 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-zul.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,779 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-twi.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:14,781 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_pcm-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,784 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ewe-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,786 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_swa-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,788 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_luo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,790 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_xho-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,792 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_twi-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,795 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_sna-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,797 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bam-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,799 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bbj-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,801 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_lug-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,803 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_hau-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,805 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_yor-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,807 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_nya-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,809 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_kin-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,812 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_tsn-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,814 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_amh-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,816 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_wol-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,818 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_mos-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,820 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_zul-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,823 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ibo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,825 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_fon-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:14,828 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-nya.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,830 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-mos.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,832 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bam.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,834 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-swa.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,837 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-sna.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,839 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-ewe.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,842 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-wol.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,844 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-pcm.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,846 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-luo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,848 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-kin.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,850 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-hau.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,852 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,854 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-tsn.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,857 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bbj.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,859 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-amh.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,861 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-yor.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,863 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-xho.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,865 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-fon.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,867 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-lug.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,869 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-zul.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,872 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-twi.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:14,874 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_pcm-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,877 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ewe-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,879 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_swa-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,881 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_luo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,883 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_xho-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,886 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_twi-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,888 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_sna-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,890 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bam-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,893 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bbj-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,895 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_lug-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,897 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_hau-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,900 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_yor-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,902 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_nya-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,904 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_kin-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,906 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_tsn-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,908 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_amh-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,911 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_wol-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,913 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_mos-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,915 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_zul-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,917 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ibo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,920 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_fon-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:14,922 copying lm_eval/tasks/afrobench/sib/sib.yaml -> build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T14:29:14,924 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,926 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,928 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,930 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,933 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,935 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,937 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,939 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,942 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,944 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,946 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,948 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,951 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,953 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,955 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,957 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,960 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,962 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,965 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,968 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,970 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,973 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,975 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,978 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,980 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,982 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,985 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,987 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,989 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,992 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,994 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,996 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:14,998 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,001 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,003 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,005 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,007 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,009 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,011 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,014 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,016 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,018 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,020 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,022 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,024 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,027 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,029 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,031 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,033 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,035 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,037 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,039 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,041 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,043 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,046 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,048 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:15,050 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,052 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,055 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,057 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,059 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,061 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,064 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,066 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,068 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,070 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,073 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,075 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,077 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,079 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,081 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,084 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,086 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,088 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,091 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,093 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,095 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,097 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,099 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,101 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,103 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,106 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,108 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,110 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,112 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,114 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,117 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,119 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,121 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,124 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,126 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,128 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,130 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,132 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,134 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,136 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,139 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,141 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,143 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,145 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,148 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,150 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,152 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,155 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,157 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,159 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,161 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,163 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,165 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,168 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,170 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,172 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:15,174 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,176 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,178 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,180 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,182 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,185 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,187 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,189 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,192 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,194 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,196 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,198 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,201 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,203 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,205 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,207 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,209 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,212 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,214 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,216 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,218 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,221 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,223 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,225 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,227 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,229 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,231 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,233 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,235 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,238 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,240 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,242 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,244 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,246 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,248 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,251 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,253 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,255 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,257 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,259 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,261 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,263 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,265 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,267 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,270 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,272 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,274 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,276 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,278 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,281 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,283 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,285 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,287 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,290 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,292 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,294 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:15,296 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,298 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,300 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,303 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,305 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,307 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,309 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,311 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,313 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,315 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,318 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,320 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,322 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,324 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,327 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,329 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,331 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,333 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,335 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,338 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,340 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,342 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,344 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,347 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,349 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,351 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,353 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,355 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,358 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,360 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,362 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,364 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,366 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,368 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,370 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,372 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,374 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,376 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,378 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,380 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,382 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,384 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,386 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,389 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,391 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,393 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,395 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,397 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,399 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,402 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,404 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,406 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,408 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,410 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,413 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,415 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:15,417 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,419 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,421 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,424 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,426 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,428 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,430 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,432 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,434 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,437 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,439 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,441 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,443 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,445 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,448 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,450 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,452 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,454 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,456 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,459 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,461 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,463 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,465 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,468 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,470 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,472 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,474 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,476 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,479 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,481 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,483 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,485 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,488 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,490 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,492 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,494 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,496 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,499 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,501 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,503 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,505 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,507 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,509 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,511 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,513 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,515 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,517 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,519 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,521 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,524 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,526 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,528 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,530 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,532 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,534 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,537 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:15,539 copying lm_eval/tasks/afrobench/masakhaner/masakhaner.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:29:15,542 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,544 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,546 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,549 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,551 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,554 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,556 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,559 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,561 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,563 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,565 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,567 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,569 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,571 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,573 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,575 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,577 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,579 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,581 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,584 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:15,586 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,588 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,590 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,592 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,594 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,597 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,599 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,601 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,603 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,605 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,608 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,610 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,612 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,614 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,617 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,619 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,621 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,623 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,625 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,627 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:15,629 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,631 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,633 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,636 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,638 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,640 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,643 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,645 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,647 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,649 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,651 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,653 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,655 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,658 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,660 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,662 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,665 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,667 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,670 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,672 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:15,674 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,676 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,679 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,681 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,683 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,685 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,687 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,689 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,692 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,694 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,696 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,698 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,700 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,703 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,705 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,707 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,709 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,712 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,714 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,717 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:15,719 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,721 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,724 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,726 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,729 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,731 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,733 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,735 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,737 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,740 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,742 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,744 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,747 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,749 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,751 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,753 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,756 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,758 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,760 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,762 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:15,765 copying lm_eval/tasks/afrobench/nollysenti/nollysenti.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti 2026-05-11T14:29:15,768 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:15,770 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:15,772 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:15,774 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:15,776 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:15,779 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:15,781 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:15,783 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:15,785 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:15,788 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:15,790 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:15,792 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:15,795 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:15,797 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:15,799 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:15,801 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:15,803 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:15,806 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:15,808 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:15,811 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:15,814 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:15,818 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:15,820 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:15,822 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:15,824 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:15,827 copying lm_eval/tasks/afrobench/afriqa/afriqa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T14:29:15,830 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,832 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,835 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,837 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,839 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,841 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,844 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,846 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,848 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:15,851 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,854 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,856 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,859 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,861 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,863 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,865 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,868 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,870 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:15,872 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,874 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,876 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,878 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,881 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,883 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,885 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,887 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,890 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:15,892 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,894 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,896 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,898 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,900 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,903 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,905 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,907 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,909 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:15,911 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,914 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,916 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,918 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,921 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,923 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,925 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,927 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,930 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:15,932 copying lm_eval/tasks/afrobench/injongointent/injongointent.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T14:29:15,934 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,937 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,939 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,942 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,944 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,946 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,948 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,950 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,952 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,955 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,957 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,959 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,961 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,964 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,966 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,968 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,970 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:15,973 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,975 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,978 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,980 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,982 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,984 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,986 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,989 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,991 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,993 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,995 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:15,997 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:16,000 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:16,002 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:16,004 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:16,006 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:16,009 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:16,011 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,013 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,015 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,017 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,019 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,022 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,024 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,026 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,029 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,032 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,035 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,038 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,040 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,043 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,045 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,047 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,049 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:16,051 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,053 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,055 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,057 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,059 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,062 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,064 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,066 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,068 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,071 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,073 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,075 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,077 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,079 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,082 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,084 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,086 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:16,089 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,091 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,093 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,095 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,097 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,100 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,102 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,104 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,106 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,108 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,110 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,113 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,115 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,117 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,119 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,122 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,124 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:16,126 copying lm_eval/tasks/afrobench/salt/salt.yaml -> build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T14:29:16,128 creating build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,129 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_nyn-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,132 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lug.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,134 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_ibo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,136 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_teo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,138 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,140 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_lgg-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,143 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_lug-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,145 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_swa-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,148 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-swa.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,151 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-teo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,153 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-nyn.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,156 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_ach-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,159 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lgg.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,161 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ach.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:16,164 creating build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,165 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_nyn-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,168 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lug.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,170 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_ibo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,172 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_teo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,174 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,176 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_lgg-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,179 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_lug-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,181 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_swa-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,184 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-swa.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,187 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-teo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,189 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-nyn.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,192 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_ach-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,194 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lgg.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,197 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ach.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:16,199 creating build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,200 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_nyn-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,202 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lug.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,204 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_ibo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,207 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_teo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,209 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,211 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_lgg-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,213 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_lug-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,215 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_swa-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,217 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-swa.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,220 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-teo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,223 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-nyn.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,225 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_ach-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,227 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lgg.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,229 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ach.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:16,231 copying lm_eval/tasks/afrobench/masakhanews/masakhanews.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:29:16,234 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,236 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,238 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,240 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,242 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,244 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,247 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,249 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,251 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,253 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,256 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,258 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,261 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,264 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,267 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,269 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:16,271 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,273 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,276 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,278 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,280 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,282 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,285 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,287 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,289 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,291 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,293 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,295 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,297 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,300 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,302 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,304 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:16,306 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,309 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,311 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,313 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,315 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,318 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,320 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,322 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,324 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,327 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,329 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,331 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,333 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,335 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,338 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,340 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:16,342 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,344 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,346 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,348 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,350 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,352 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,355 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,357 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,359 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,361 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,363 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,365 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,367 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,369 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,371 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,373 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:16,376 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,378 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,380 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,382 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,384 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,387 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,389 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,391 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,393 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,395 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,398 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,400 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,402 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,404 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,406 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,408 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:16,410 copying lm_eval/tasks/xquad/xquad_th.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,412 copying lm_eval/tasks/xquad/xquad_el.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,414 copying lm_eval/tasks/xquad/xquad_en.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,416 copying lm_eval/tasks/xquad/xquad_de.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,418 copying lm_eval/tasks/xquad/xquad_ro.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,420 copying lm_eval/tasks/xquad/xquad_vi.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,422 copying lm_eval/tasks/xquad/xquad_ar.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,424 copying lm_eval/tasks/xquad/xquad_zh.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,427 copying lm_eval/tasks/xquad/xquad_ru.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,429 copying lm_eval/tasks/xquad/xquad_hi.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,431 copying lm_eval/tasks/xquad/xquad_tr.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,433 copying lm_eval/tasks/xquad/xquad_es.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:16,435 copying lm_eval/tasks/spanish_bench/spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,437 copying lm_eval/tasks/spanish_bench/openbookqa_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,439 copying lm_eval/tasks/spanish_bench/paws_es_spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,442 copying lm_eval/tasks/spanish_bench/cocoteros_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,444 copying lm_eval/tasks/spanish_bench/xlsum_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,446 copying lm_eval/tasks/spanish_bench/copa_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,448 copying lm_eval/tasks/spanish_bench/escola.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,450 copying lm_eval/tasks/spanish_bench/wnli_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,452 copying lm_eval/tasks/spanish_bench/mgsm_direct_es_spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,454 copying lm_eval/tasks/spanish_bench/xnli_es_spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:16,456 creating build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:16,457 copying lm_eval/tasks/spanish_bench/phrases_es/phrases_es-va.yaml -> build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:16,460 copying lm_eval/tasks/spanish_bench/phrases_es/phrases_va-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:16,462 copying lm_eval/tasks/spanish_bench/flores_es/flores_eu-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,464 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-ca.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,466 copying lm_eval/tasks/spanish_bench/flores_es/flores_ca-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,468 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-it.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,470 copying lm_eval/tasks/spanish_bench/flores_es/flores_es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,472 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-gl.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,474 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-fr.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,476 copying lm_eval/tasks/spanish_bench/flores_es/flores_pt-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,478 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-eu.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,480 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-de.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,482 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-pt.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,485 copying lm_eval/tasks/spanish_bench/flores_es/flores_gl-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,487 copying lm_eval/tasks/spanish_bench/flores_es/flores_fr-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,489 copying lm_eval/tasks/spanish_bench/flores_es/flores_de-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,491 copying lm_eval/tasks/spanish_bench/flores_es/flores_en-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,493 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-en.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,495 copying lm_eval/tasks/spanish_bench/flores_es/flores_it-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:16,498 copying lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,500 copying lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,502 copying lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,504 copying lm_eval/tasks/paloma/paloma_wikitext_103.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,507 copying lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,509 copying lm_eval/tasks/paloma/paloma_ptb.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,511 copying lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,513 copying lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,515 copying lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,517 copying lm_eval/tasks/paloma/paloma_c4_100_domains.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,520 copying lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,522 copying lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,524 copying lm_eval/tasks/paloma/paloma_c4_en.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,526 copying lm_eval/tasks/paloma/paloma_mc4.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,528 copying lm_eval/tasks/paloma/paloma_gab.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,531 copying lm_eval/tasks/paloma/paloma_redpajama.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:16,533 copying lm_eval/tasks/med_text_classification/med_text_classification_easy.yaml -> build/lib/lm_eval/tasks/med_text_classification 2026-05-11T14:29:16,535 copying lm_eval/tasks/med_text_classification/med_text_classification_hard.yaml -> build/lib/lm_eval/tasks/med_text_classification 2026-05-11T14:29:16,538 copying lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,540 copying lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,542 copying lm_eval/tasks/translation/wmt16_de-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,544 copying lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,546 copying lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,548 copying lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,550 copying lm_eval/tasks/translation/wmt16_en-de.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,552 copying lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:16,554 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_gl.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,556 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_ca.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,558 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_es.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,560 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_es.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,562 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_es.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,564 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_en.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,566 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_eu.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,568 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_gl.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,570 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_en.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,572 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_eu.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,574 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_ca.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,576 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_eu.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,579 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_gl.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,581 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_ca.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,583 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_en.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:16,585 copying lm_eval/tasks/cabbq/cabbq_age.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,587 copying lm_eval/tasks/cabbq/cabbq_physical_appearance.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,589 copying lm_eval/tasks/cabbq/cabbq_race_ethnicity.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,592 copying lm_eval/tasks/cabbq/cabbq_nationality.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,594 copying lm_eval/tasks/cabbq/cabbq_spanish_region.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,596 copying lm_eval/tasks/cabbq/cabbq_gender.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,598 copying lm_eval/tasks/cabbq/cabbq_ses.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,601 copying lm_eval/tasks/cabbq/cabbq_disability_status.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,603 copying lm_eval/tasks/cabbq/cabbq_religion.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,605 copying lm_eval/tasks/cabbq/cabbq.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,607 copying lm_eval/tasks/cabbq/cabbq_lgbtqia.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:16,610 copying lm_eval/tasks/common_voice/common_voice_en.yaml -> build/lib/lm_eval/tasks/common_voice 2026-05-11T14:29:16,612 copying lm_eval/tasks/groundcocoa/groundcocoa.yaml -> build/lib/lm_eval/tasks/groundcocoa 2026-05-11T14:29:16,614 copying lm_eval/tasks/fda/fda.yaml -> build/lib/lm_eval/tasks/fda 2026-05-11T14:29:16,616 copying lm_eval/tasks/squadv2/squadv2.yaml -> build/lib/lm_eval/tasks/squadv2 2026-05-11T14:29:16,618 copying lm_eval/tasks/aime/aime.yaml -> build/lib/lm_eval/tasks/aime 2026-05-11T14:29:16,620 copying lm_eval/tasks/aime/aime24.yaml -> build/lib/lm_eval/tasks/aime 2026-05-11T14:29:16,622 copying lm_eval/tasks/aime/aime25.yaml -> build/lib/lm_eval/tasks/aime 2026-05-11T14:29:16,624 copying lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,627 copying lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,629 copying lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,631 copying lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,633 copying lm_eval/tasks/french_bench/french_bench_xnli.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,635 copying lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,637 copying lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,639 copying lm_eval/tasks/french_bench/french_bench_hellaswag.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,641 copying lm_eval/tasks/french_bench/french_bench_vocab.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,644 copying lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,646 copying lm_eval/tasks/french_bench/french_bench_grammar.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,648 copying lm_eval/tasks/french_bench/french_bench_fquadv2.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,650 copying lm_eval/tasks/french_bench/french_bench_trivia.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,652 copying lm_eval/tasks/french_bench/french_bench_multifquad.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,655 copying lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,657 copying lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,659 copying lm_eval/tasks/french_bench/french_bench_reading_comp.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,662 copying lm_eval/tasks/french_bench/french_bench_boolqa.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:16,664 copying lm_eval/tasks/eq_bench/default.yaml -> build/lib/lm_eval/tasks/eq_bench 2026-05-11T14:29:16,666 copying lm_eval/tasks/eq_bench/multilingual/eqbench_es.yaml -> build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:29:16,669 copying lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml -> build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:29:16,671 copying lm_eval/tasks/esbbq/esbbq_spanish_region.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,673 copying lm_eval/tasks/esbbq/esbbq_race_ethnicity.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,676 copying lm_eval/tasks/esbbq/esbbq_physical_appearance.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,678 copying lm_eval/tasks/esbbq/esbbq_ses.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,680 copying lm_eval/tasks/esbbq/esbbq_religion.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,682 copying lm_eval/tasks/esbbq/esbbq_nationality.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,684 copying lm_eval/tasks/esbbq/esbbq_disability_status.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,686 copying lm_eval/tasks/esbbq/esbbq.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,688 copying lm_eval/tasks/esbbq/esbbq_age.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,690 copying lm_eval/tasks/esbbq/esbbq_lgbtqia.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,692 copying lm_eval/tasks/esbbq/esbbq_gender.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:16,694 creating build/lib/lm_eval/tasks/arc 2026-05-11T14:29:16,695 copying lm_eval/tasks/arc/arc_easy.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T14:29:16,697 copying lm_eval/tasks/arc/arc_challenge.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T14:29:16,699 copying lm_eval/tasks/arc/arc_challenge_chat.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T14:29:16,702 copying lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/lib/lm_eval/tasks/realtoxicityprompts 2026-05-11T14:29:16,704 creating build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,705 copying lm_eval/tasks/bhs/bhs_hindi.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,707 copying lm_eval/tasks/bhs/hindi-S_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,709 copying lm_eval/tasks/bhs/basque-S-S_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,712 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_ni_A.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,714 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V1_V2.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,717 copying lm_eval/tasks/bhs/hindi-S_ne_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,719 copying lm_eval/tasks/bhs/basque-S-IO_S_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,721 copying lm_eval/tasks/bhs/basque-S-S_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,724 copying lm_eval/tasks/bhs/basque-DO-S_IO_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,726 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_ni_AN.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,728 copying lm_eval/tasks/bhs/bhs_swahili.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,731 copying lm_eval/tasks/bhs/swahili-N_of_Poss_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,733 copying lm_eval/tasks/bhs/hindi-S_ne_PossPRN_PossN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,736 copying lm_eval/tasks/bhs/swahili-N_of_Poss_ni_A.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,738 copying lm_eval/tasks/bhs/basque-S-S_IO_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,740 copying lm_eval/tasks/bhs/bhs_basque.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,742 copying lm_eval/tasks/bhs/basque-DO-S_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,744 copying lm_eval/tasks/bhs/basque-IO-IO_S_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,747 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_V_ni_AN.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,749 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,751 copying lm_eval/tasks/bhs/hindi-S_ne_PossPRN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,753 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,755 copying lm_eval/tasks/bhs/hindi-S_PossPRN_PossN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,757 copying lm_eval/tasks/bhs/basque-IO-S_IO_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,760 copying lm_eval/tasks/bhs/hindi-S_PossPRN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:16,761 creating build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,763 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,765 copying lm_eval/tasks/mmlu-redux/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,767 copying lm_eval/tasks/mmlu-redux/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,769 copying lm_eval/tasks/mmlu-redux/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,771 copying lm_eval/tasks/mmlu-redux/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,773 copying lm_eval/tasks/mmlu-redux/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,775 copying lm_eval/tasks/mmlu-redux/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,777 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,779 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,781 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,783 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,785 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,787 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,790 copying lm_eval/tasks/mmlu-redux/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,792 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,794 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,796 copying lm_eval/tasks/mmlu-redux/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,798 copying lm_eval/tasks/mmlu-redux/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,801 copying lm_eval/tasks/mmlu-redux/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,803 copying lm_eval/tasks/mmlu-redux/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,805 copying lm_eval/tasks/mmlu-redux/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,807 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,809 copying lm_eval/tasks/mmlu-redux/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,812 copying lm_eval/tasks/mmlu-redux/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,814 copying lm_eval/tasks/mmlu-redux/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,816 copying lm_eval/tasks/mmlu-redux/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,818 copying lm_eval/tasks/mmlu-redux/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,821 copying lm_eval/tasks/mmlu-redux/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,823 copying lm_eval/tasks/mmlu-redux/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,825 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,827 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,829 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,831 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,833 copying lm_eval/tasks/mmlu-redux/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,836 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,838 copying lm_eval/tasks/mmlu-redux/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,840 copying lm_eval/tasks/mmlu-redux/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,842 copying lm_eval/tasks/mmlu-redux/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,844 copying lm_eval/tasks/mmlu-redux/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,847 copying lm_eval/tasks/mmlu-redux/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,849 copying lm_eval/tasks/mmlu-redux/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,851 copying lm_eval/tasks/mmlu-redux/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,853 copying lm_eval/tasks/mmlu-redux/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,855 copying lm_eval/tasks/mmlu-redux/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,858 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,860 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,862 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,864 copying lm_eval/tasks/mmlu-redux/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,866 copying lm_eval/tasks/mmlu-redux/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,869 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,871 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,873 copying lm_eval/tasks/mmlu-redux/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,875 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,877 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,879 copying lm_eval/tasks/mmlu-redux/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,882 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,883 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,885 copying lm_eval/tasks/mmlu-redux/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:16,887 creating build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T14:29:16,888 copying lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T14:29:16,890 copying lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T14:29:16,893 copying lm_eval/tasks/webqs/webqs.yaml -> build/lib/lm_eval/tasks/webqs 2026-05-11T14:29:16,894 creating build/lib/lm_eval/tasks/benchmarks 2026-05-11T14:29:16,895 copying lm_eval/tasks/benchmarks/pythia.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T14:29:16,898 copying lm_eval/tasks/benchmarks/minerva_math.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T14:29:16,900 copying lm_eval/tasks/benchmarks/t0_eval.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T14:29:16,902 copying lm_eval/tasks/benchmarks/openllm.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T14:29:16,904 creating build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:16,905 copying lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:16,908 copying lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:16,910 creating build/lib/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T14:29:16,912 copying lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/lib/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T14:29:16,914 creating build/lib/lm_eval/tasks/lambada 2026-05-11T14:29:16,915 copying lm_eval/tasks/lambada/lambada_openai.yaml -> build/lib/lm_eval/tasks/lambada 2026-05-11T14:29:16,917 copying lm_eval/tasks/lambada/lambada_standard.yaml -> build/lib/lm_eval/tasks/lambada 2026-05-11T14:29:16,920 creating build/lib/lm_eval/tasks/gsm8k 2026-05-11T14:29:16,921 copying lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T14:29:16,923 copying lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T14:29:16,926 copying lm_eval/tasks/gsm8k/gsm8k.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T14:29:16,928 copying lm_eval/tasks/gsm8k/gsm8k-cot-llama.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T14:29:16,930 copying lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T14:29:16,933 copying lm_eval/tasks/mts_dialog/mts_dialog_perplexity.yaml -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T14:29:16,935 copying lm_eval/tasks/mts_dialog/mts_dialog.yaml -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T14:29:16,937 copying lm_eval/tasks/darijammlu/darijammlu_biology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,939 copying lm_eval/tasks/darijammlu/darijammlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,942 copying lm_eval/tasks/darijammlu/darijammlu_security_studies.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,944 copying lm_eval/tasks/darijammlu/_darijammlu_ar_mmlu.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,946 copying lm_eval/tasks/darijammlu/darijammlu_human_aging.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,948 copying lm_eval/tasks/darijammlu/darijammlu_political_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,950 copying lm_eval/tasks/darijammlu/darijammlu_natural_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,952 copying lm_eval/tasks/darijammlu/darijammlu_islamic_studies.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,955 copying lm_eval/tasks/darijammlu/darijammlu_computer_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,957 copying lm_eval/tasks/darijammlu/darijammlu_geography.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,959 copying lm_eval/tasks/darijammlu/darijammlu_general_knowledge.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,961 copying lm_eval/tasks/darijammlu/darijammlu_history.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,963 copying lm_eval/tasks/darijammlu/darijammlu_math.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,966 copying lm_eval/tasks/darijammlu/darijammlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,968 copying lm_eval/tasks/darijammlu/darijammlu_driving_test.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,970 copying lm_eval/tasks/darijammlu/darijammlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,972 copying lm_eval/tasks/darijammlu/darijammlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,974 copying lm_eval/tasks/darijammlu/darijammlu_arabic_language_(grammar).yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,976 copying lm_eval/tasks/darijammlu/darijammlu_world_religions.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,978 copying lm_eval/tasks/darijammlu/darijammlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,980 copying lm_eval/tasks/darijammlu/darijammlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,982 copying lm_eval/tasks/darijammlu/darijammlu_philosophy_ar.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,984 copying lm_eval/tasks/darijammlu/darijammlu_global_facts.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,986 copying lm_eval/tasks/darijammlu/darijammlu_accounting.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,988 copying lm_eval/tasks/darijammlu/darijammlu_sociology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,990 copying lm_eval/tasks/darijammlu/darijammlu_public_relations.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,992 copying lm_eval/tasks/darijammlu/darijammlu_law.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,994 copying lm_eval/tasks/darijammlu/darijammlu_management_ar.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,996 copying lm_eval/tasks/darijammlu/darijammlu_management.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:16,999 copying lm_eval/tasks/darijammlu/darijammlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,001 copying lm_eval/tasks/darijammlu/_darijammlu.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,003 copying lm_eval/tasks/darijammlu/darijammlu_philosophy.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,006 copying lm_eval/tasks/darijammlu/darijammlu_economics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,008 copying lm_eval/tasks/darijammlu/darijammlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,010 copying lm_eval/tasks/darijammlu/darijammlu_nutrition.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,013 copying lm_eval/tasks/darijammlu/darijammlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,015 copying lm_eval/tasks/darijammlu/darijammlu_social_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,017 copying lm_eval/tasks/darijammlu/darijammlu_professional_law.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,020 copying lm_eval/tasks/darijammlu/darijammlu_international_law.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,022 copying lm_eval/tasks/darijammlu/darijammlu_arabic_language_(general).yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,024 copying lm_eval/tasks/darijammlu/darijammlu_arabic_language.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,026 copying lm_eval/tasks/darijammlu/darijammlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,028 copying lm_eval/tasks/darijammlu/darijammlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,030 copying lm_eval/tasks/darijammlu/_darijammlu_mmlu.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,032 copying lm_eval/tasks/darijammlu/darijammlu_civics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,035 copying lm_eval/tasks/darijammlu/darijammlu_marketing.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,037 copying lm_eval/tasks/darijammlu/darijammlu_physics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:17,039 copying lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T14:29:17,041 copying lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T14:29:17,043 copying lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T14:29:17,046 copying lm_eval/tasks/meddialog/meddialog_qsumm_perplexity.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T14:29:17,048 copying lm_eval/tasks/meddialog/meddialog_qsumm.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T14:29:17,050 copying lm_eval/tasks/meddialog/meddialog_raw_dialogues.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T14:29:17,052 copying lm_eval/tasks/meddialog/meddialog_raw_perplexity.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T14:29:17,054 copying lm_eval/tasks/histoires_morales/histoires_morales.yaml -> build/lib/lm_eval/tasks/histoires_morales 2026-05-11T14:29:17,056 creating build/lib/lm_eval/tasks/gsm_plus 2026-05-11T14:29:17,058 copying lm_eval/tasks/gsm_plus/gsm_plus_mini.yaml -> build/lib/lm_eval/tasks/gsm_plus 2026-05-11T14:29:17,060 copying lm_eval/tasks/gsm_plus/gsm_plus.yaml -> build/lib/lm_eval/tasks/gsm_plus 2026-05-11T14:29:17,062 copying lm_eval/tasks/darijahellaswag/darijahellaswag.yaml -> build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T14:29:17,064 creating build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:17,065 copying lm_eval/tasks/kormedmcqa/_kormedmcqa.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:17,068 copying lm_eval/tasks/kormedmcqa/dentist.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:17,070 copying lm_eval/tasks/kormedmcqa/nurse.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:17,072 copying lm_eval/tasks/kormedmcqa/pharm.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:17,074 copying lm_eval/tasks/kormedmcqa/doctor.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:17,076 creating build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,077 copying lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,080 copying lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,082 copying lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,084 copying lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,086 copying lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,089 copying lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,091 copying lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,093 copying lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,096 copying lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,098 copying lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,100 copying lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,102 copying lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:17,104 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_economics.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,106 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_business.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,108 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_math.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,110 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_law.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,112 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,114 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_biology.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,116 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,118 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_health.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,120 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_other.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,123 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_history.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,125 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,128 copying lm_eval/tasks/mmlu-pro-plus/_mmlu_pro_plus.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,130 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,132 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_physics.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,134 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_engineering.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:17,136 copying lm_eval/tasks/wsc273/default.yaml -> build/lib/lm_eval/tasks/wsc273 2026-05-11T14:29:17,138 creating build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,140 copying lm_eval/tasks/turblimp/ellipsis.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,142 copying lm_eval/tasks/turblimp/suspended_affixation.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,144 copying lm_eval/tasks/turblimp/scrambling.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,146 copying lm_eval/tasks/turblimp/nominalization.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,148 copying lm_eval/tasks/turblimp/quantifiers.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,151 copying lm_eval/tasks/turblimp/relative_clauses.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,153 copying lm_eval/tasks/turblimp/island_effects.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,155 copying lm_eval/tasks/turblimp/binding.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,157 copying lm_eval/tasks/turblimp/irregular_forms.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,160 copying lm_eval/tasks/turblimp/turblimp_group.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,162 copying lm_eval/tasks/turblimp/argument_structure_transitive.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,164 copying lm_eval/tasks/turblimp/subject_agreement.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,166 copying lm_eval/tasks/turblimp/argument_structure_ditransitive.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,168 copying lm_eval/tasks/turblimp/determiners.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,171 copying lm_eval/tasks/turblimp/anaphor_agreement.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,173 copying lm_eval/tasks/turblimp/npi_licensing.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,175 copying lm_eval/tasks/turblimp/passives.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:17,176 creating build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:17,177 copying lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:17,180 copying lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:17,181 copying lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:17,183 copying lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:17,186 copying lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:17,188 copying lm_eval/tasks/score/score_robustness.yaml -> build/lib/lm_eval/tasks/score 2026-05-11T14:29:17,190 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_math.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,192 copying lm_eval/tasks/score/agi_eval/score_option_order_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,195 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_lr.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,198 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_ar.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,200 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_ar.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,202 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_aqua_rat.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,204 copying lm_eval/tasks/score/agi_eval/score_prompt_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,207 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_logiqa_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,209 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_logiqa_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,212 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lsat_rc.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,214 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,216 copying lm_eval/tasks/score/agi_eval/score_non_greedy_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,219 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_aqua_rat.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,221 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,223 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_lr.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,225 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lsat_rc.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,228 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_lr.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,230 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_ar.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,232 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_rc.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,234 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,236 copying lm_eval/tasks/score/agi_eval/score_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,239 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_math.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,241 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_math.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,243 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_logiqa_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,245 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_aqua_rat.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:17,248 copying lm_eval/tasks/score/math/prompt_robustness_math_geometry.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,250 copying lm_eval/tasks/score/math/prompt_robustness_math_precalc.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,252 copying lm_eval/tasks/score/math/score_prompt_robustness_math.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,254 copying lm_eval/tasks/score/math/non_greedy_robustness_math_num_theory.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,257 copying lm_eval/tasks/score/math/non_greedy_robustness_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,259 copying lm_eval/tasks/score/math/non_greedy_robustness_math_geometry.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,261 copying lm_eval/tasks/score/math/score_non_greedy_robustness_math.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,263 copying lm_eval/tasks/score/math/prompt_robustness_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,265 copying lm_eval/tasks/score/math/score_robustness_math.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,268 copying lm_eval/tasks/score/math/non_greedy_robustness_math_precalc.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,270 copying lm_eval/tasks/score/math/non_greedy_robustness_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,273 copying lm_eval/tasks/score/math/prompt_robustness_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,275 copying lm_eval/tasks/score/math/non_greedy_robustness_math_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,277 copying lm_eval/tasks/score/math/prompt_robustness_math_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,279 copying lm_eval/tasks/score/math/prompt_robustness_math_prealgebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,282 copying lm_eval/tasks/score/math/non_greedy_robustness_math_prealgebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,284 copying lm_eval/tasks/score/math/prompt_robustness_math_num_theory.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:17,287 copying lm_eval/tasks/score/mmlu_pro/score_non_greedy_robustness_mmlu_pro.yaml -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T14:29:17,289 copying lm_eval/tasks/score/mmlu_pro/score_option_order_robustness_mmlu_pro.yaml -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T14:29:17,292 copying lm_eval/tasks/score/mmlu_pro/score_prompt_robustness_mmlu_pro.yaml -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T14:29:17,294 creating build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,296 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,299 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,302 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,305 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,308 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,310 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,313 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,316 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,318 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,321 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,323 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,326 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,329 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,332 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,334 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,337 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,340 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,342 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,345 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,347 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,350 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,352 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,355 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,357 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,360 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,362 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,364 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,367 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,369 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,372 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,374 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,377 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,380 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,382 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,384 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,387 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,389 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,391 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,394 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,396 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,399 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,401 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,403 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,406 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,408 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,411 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,413 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,416 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,419 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,421 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,425 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,427 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,430 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,432 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,435 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,437 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,440 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,442 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:17,444 creating build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,445 copying lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,448 copying lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,450 copying lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,452 copying lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,454 copying lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,456 copying lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,458 copying lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,461 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,463 copying lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,465 copying lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,467 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,470 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,472 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,474 copying lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,476 copying lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,479 copying lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,481 copying lm_eval/tasks/mmlu/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,484 copying lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,486 copying lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,489 copying lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,491 copying lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,493 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,496 copying lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,498 copying lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,500 copying lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,502 copying lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,505 copying lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,507 copying lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,509 copying lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,511 copying lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,514 copying lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,516 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,518 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,520 copying lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,522 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,525 copying lm_eval/tasks/mmlu/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,527 copying lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,529 copying lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,531 copying lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,533 copying lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,535 copying lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,537 copying lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,539 copying lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,542 copying lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,544 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,546 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,548 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,550 copying lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,553 copying lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,555 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,557 copying lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,559 copying lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,562 copying lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,564 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,566 copying lm_eval/tasks/mmlu/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,568 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,571 copying lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,574 copying lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:17,578 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,580 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,582 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,584 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,587 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,589 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,591 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,594 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,596 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,598 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,600 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,603 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,605 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,607 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,610 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,612 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,614 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,616 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,619 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,621 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,623 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,625 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,628 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,630 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,632 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,634 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,636 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,639 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,641 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,643 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,646 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,648 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,651 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,653 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,655 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,657 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,660 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,662 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,664 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,666 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,669 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,671 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,673 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,675 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,677 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,680 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,682 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,684 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,686 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,689 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,691 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,693 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,696 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,698 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,700 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,703 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,705 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,708 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:17,710 creating build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,711 copying lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,714 copying lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,716 copying lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,718 copying lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,720 copying lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,723 copying lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,725 copying lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,727 copying lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,730 copying lm_eval/tasks/mmlu/default/_mmlu_stem.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,732 copying lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,735 copying lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,737 copying lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,740 copying lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,743 copying lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,746 copying lm_eval/tasks/mmlu/default/_mmlu_social_sciences.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,749 copying lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,751 copying lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,754 copying lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,757 copying lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,760 copying lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,763 copying lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,767 copying lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,770 copying lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,773 copying lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,775 copying lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,778 copying lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,781 copying lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,784 copying lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,787 copying lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,790 copying lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,792 copying lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,795 copying lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,798 copying lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,801 copying lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,803 copying lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,806 copying lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,809 copying lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,811 copying lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,814 copying lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,816 copying lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,819 copying lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,822 copying lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,824 copying lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,827 copying lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,829 copying lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,832 copying lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,835 copying lm_eval/tasks/mmlu/default/_mmlu_humanities.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,837 copying lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,839 copying lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,842 copying lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,844 copying lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,846 copying lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,849 copying lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,851 copying lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,853 copying lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,855 copying lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,858 copying lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,860 copying lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,862 copying lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,864 copying lm_eval/tasks/mmlu/default/_mmlu_other.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,867 copying lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,870 copying lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:17,872 creating build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,874 copying lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,876 copying lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,879 copying lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,881 copying lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,884 copying lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,886 copying lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,889 copying lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,891 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,893 copying lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,896 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,898 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,900 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,902 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,904 copying lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,906 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,908 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,911 copying lm_eval/tasks/mmlu/continuation/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,913 copying lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,915 copying lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,917 copying lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,920 copying lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,923 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,925 copying lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,927 copying lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,929 copying lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,932 copying lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,934 copying lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,936 copying lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,938 copying lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,941 copying lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,943 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,945 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,947 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,949 copying lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,951 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,954 copying lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,956 copying lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,958 copying lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,960 copying lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,962 copying lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,964 copying lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,966 copying lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,969 copying lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,971 copying lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,973 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,975 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,977 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,979 copying lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,982 copying lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,984 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,986 copying lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,988 copying lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,990 copying lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,992 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,995 copying lm_eval/tasks/mmlu/continuation/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,997 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:17,999 copying lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:18,002 copying lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:18,004 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,006 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,009 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,011 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,013 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,015 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,017 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,019 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,022 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,024 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,026 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,028 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,030 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,032 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,034 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,037 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,040 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,042 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,044 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,046 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,049 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,051 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,053 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,055 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,058 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,060 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,062 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,064 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,067 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,069 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,071 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,073 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,076 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,078 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,080 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,082 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,084 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,087 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,089 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,091 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,093 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,096 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,099 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,101 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,104 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,107 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,110 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,112 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,114 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,116 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,119 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,121 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,123 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,125 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,128 copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,130 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,132 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,134 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:18,136 creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,137 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,140 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,142 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,144 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,147 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,150 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,152 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,154 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,157 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,159 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,161 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,163 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,165 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,168 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,170 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,172 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,174 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,177 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,179 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,181 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,184 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,186 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,188 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,190 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,192 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,194 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,196 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,199 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,201 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,203 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,205 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,207 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,209 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,211 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,213 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,216 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,218 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,220 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,222 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,225 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,227 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,229 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,231 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,233 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,236 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,238 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,240 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,242 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,244 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,246 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,248 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,250 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,252 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,254 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,256 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,259 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,261 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,263 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:18,265 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,268 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,270 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,272 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,274 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,276 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,278 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,280 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,282 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,284 copying lm_eval/tasks/evalita_llm/_evalita-mp_ls_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,287 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,289 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,291 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,293 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,295 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,297 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,299 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,301 copying lm_eval/tasks/evalita_llm/_evalita-mp.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,303 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,305 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,307 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,309 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,311 copying lm_eval/tasks/evalita_llm/_evalita-mp_gen.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,313 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,315 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,318 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,320 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,322 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,324 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,326 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,328 copying lm_eval/tasks/evalita_llm/_evalita-mp_mc.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,330 copying lm_eval/tasks/evalita_llm/_evalita-mp_re_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,332 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,334 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,336 copying lm_eval/tasks/evalita_llm/_evalita-mp_ls_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,338 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,340 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,343 copying lm_eval/tasks/evalita_llm/_evalita-mp_ls_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,345 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,347 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,349 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,351 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,353 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,356 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,358 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,360 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,362 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,365 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,367 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,369 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,371 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,373 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,375 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,377 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,379 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,381 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,383 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,385 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,387 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,389 copying lm_eval/tasks/evalita_llm/_evalita-mp_re_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,392 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,394 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,396 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,398 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,400 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,402 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,405 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,407 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,409 copying lm_eval/tasks/evalita_llm/_evalita-mp_re_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,411 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,413 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,415 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,417 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:18,419 creating build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,421 copying lm_eval/tasks/kbl/knowledge/kbl_statute_hallucination_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,423 copying lm_eval/tasks/kbl/knowledge/kbl_statute_number_and_content_matching_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,426 copying lm_eval/tasks/kbl/knowledge/kbl_offense_component_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,428 copying lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,430 copying lm_eval/tasks/kbl/knowledge/kbl_query_statute_matching_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,433 copying lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_reasoning_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,435 copying lm_eval/tasks/kbl/knowledge/kbl_legal_concept_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:18,437 creating build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,439 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,441 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,443 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,445 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,448 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,450 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,452 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2024.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,454 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,457 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,459 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,461 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,464 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,466 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:18,468 creating build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,469 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2024.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,471 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,473 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,475 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,477 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,480 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,482 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,484 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,486 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,488 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,491 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,493 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,495 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:18,497 creating build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,498 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2010.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,500 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,502 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,504 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,506 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,508 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,510 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,512 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,514 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,516 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,518 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,521 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,523 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,525 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2011.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:18,527 creating build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,528 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2024.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,530 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,532 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,534 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,537 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,539 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,541 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,543 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,545 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,548 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,550 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,552 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,554 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2025.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,556 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:18,558 creating build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T14:29:18,559 copying lm_eval/tasks/kbl/reasoning/kbl_causal_reasoning_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T14:29:18,561 copying lm_eval/tasks/kbl/reasoning/kbl_statement_consistency_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T14:29:18,563 copying lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_q_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T14:29:18,565 copying lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_p_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T14:29:18,568 copying lm_eval/tasks/lingoly/lingoly_nocontext.yaml -> build/lib/lm_eval/tasks/lingoly 2026-05-11T14:29:18,570 copying lm_eval/tasks/lingoly/lingoly_group.yaml -> build/lib/lm_eval/tasks/lingoly 2026-05-11T14:29:18,572 copying lm_eval/tasks/lingoly/lingoly_context.yaml -> build/lib/lm_eval/tasks/lingoly 2026-05-11T14:29:18,574 copying lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,576 copying lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,578 copying lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,580 copying lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,582 copying lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,584 copying lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,586 copying lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,589 copying lm_eval/tasks/hendrycks_math/hendrycks_math500.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,591 copying lm_eval/tasks/hendrycks_math/hendrycks_math.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:18,593 creating build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:29:18,594 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p0.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:29:18,596 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p1.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:29:18,599 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p3.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:29:18,601 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p2.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:29:18,603 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p4.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:29:18,605 creating build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:29:18,606 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p4.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:29:18,608 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p1.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:29:18,610 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p0.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:29:18,612 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p2.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:29:18,614 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p3.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:29:18,617 copying lm_eval/tasks/noreval/norquad/norquad_p1.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:29:18,619 copying lm_eval/tasks/noreval/norquad/norquad_p0.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:29:18,621 copying lm_eval/tasks/noreval/norquad/norquad_p2.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:29:18,623 copying lm_eval/tasks/noreval/norquad/norquad_p3.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:29:18,624 copying lm_eval/tasks/noreval/norquad/norquad_p4.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:29:18,626 creating build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:29:18,627 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:29:18,630 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:29:18,632 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:29:18,634 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:29:18,636 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:29:18,638 creating build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:29:18,639 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:29:18,641 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:29:18,643 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:29:18,645 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:29:18,648 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:29:18,650 creating build/lib/lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T14:29:18,651 copying lm_eval/tasks/noreval/norrewrite-instruct/norrewrite_instruct.yaml -> build/lib/lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T14:29:18,653 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:29:18,654 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:29:18,657 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:29:18,659 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:29:18,661 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:29:18,663 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:29:18,664 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:29:18,666 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:29:18,668 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:29:18,670 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:29:18,672 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:29:18,673 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:29:18,675 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:29:18,677 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:29:18,680 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:29:18,681 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:29:18,682 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:29:18,685 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:29:18,687 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:29:18,689 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:29:18,691 creating build/lib/lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T14:29:18,692 copying lm_eval/tasks/noreval/norsummarize-instruct/norsummarize_instruct.yaml -> build/lib/lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T14:29:18,694 creating build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:29:18,695 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:29:18,698 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:29:18,700 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:29:18,702 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:29:18,704 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:29:18,706 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p5.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:29:18,708 creating build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:29:18,709 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:29:18,711 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:29:18,714 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:29:18,716 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:29:18,718 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:29:18,720 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p5.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:29:18,722 creating build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:29:18,724 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:29:18,726 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:29:18,728 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:29:18,730 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:29:18,732 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:29:18,734 creating build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:29:18,735 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:29:18,737 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:29:18,739 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:29:18,741 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:29:18,743 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:29:18,745 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p0.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:29:18,748 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p4.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:29:18,750 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p2.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:29:18,752 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p1.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:29:18,754 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p3.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:29:18,756 creating build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:29:18,757 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p2.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:29:18,759 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p0.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:29:18,761 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p1.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:29:18,763 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p3.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:29:18,765 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p4.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:29:18,767 creating build/lib/lm_eval/tasks/noreval/ncb 2026-05-11T14:29:18,768 copying lm_eval/tasks/noreval/ncb/ncb.yaml -> build/lib/lm_eval/tasks/noreval/ncb 2026-05-11T14:29:18,770 creating build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:29:18,771 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:29:18,774 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:29:18,776 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:29:18,778 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:29:18,780 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:29:18,782 creating build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:29:18,783 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:29:18,785 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:29:18,787 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:29:18,789 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:29:18,791 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:29:18,794 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:29:18,796 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:29:18,798 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:29:18,800 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:29:18,802 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:29:18,804 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:29:18,806 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:29:18,809 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:29:18,811 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:29:18,813 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:29:18,815 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:29:18,816 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:29:18,818 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:29:18,820 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:29:18,822 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:29:18,824 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:29:18,826 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:29:18,827 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:29:18,830 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:29:18,832 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:29:18,834 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:29:18,837 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:29:18,839 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:29:18,841 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:29:18,843 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:29:18,845 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:29:18,847 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:29:18,850 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:29:18,852 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:29:18,854 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:29:18,856 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:29:18,858 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:29:18,861 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,863 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,866 copying lm_eval/tasks/mmlusr/question_only/question_only_world_religions.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,868 copying lm_eval/tasks/mmlusr/question_only/question_only_human_aging.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,870 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,872 copying lm_eval/tasks/mmlusr/question_only/question_only_nutrition.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,874 copying lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,876 copying lm_eval/tasks/mmlusr/question_only/question_only_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,879 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,881 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,883 copying lm_eval/tasks/mmlusr/question_only/question_only_marketing.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,885 copying lm_eval/tasks/mmlusr/question_only/question_only_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,887 copying lm_eval/tasks/mmlusr/question_only/question_only_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,890 copying lm_eval/tasks/mmlusr/question_only/question_only_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,892 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,894 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,896 copying lm_eval/tasks/mmlusr/question_only/question_only_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,898 copying lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,901 copying lm_eval/tasks/mmlusr/question_only/question_only_management.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,903 copying lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,905 copying lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,907 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,909 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,911 copying lm_eval/tasks/mmlusr/question_only/question_only_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,913 copying lm_eval/tasks/mmlusr/question_only/question_only_global_facts.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,916 copying lm_eval/tasks/mmlusr/question_only/question_only_sociology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,918 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,920 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,922 copying lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,924 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,927 copying lm_eval/tasks/mmlusr/question_only/question_only_public_relations.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,929 copying lm_eval/tasks/mmlusr/question_only/question_only_security_studies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,931 copying lm_eval/tasks/mmlusr/question_only/question_only_international_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,933 copying lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,935 copying lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,937 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,940 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,942 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,944 copying lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,946 copying lm_eval/tasks/mmlusr/question_only/question_only_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,949 copying lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,951 copying lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,953 copying lm_eval/tasks/mmlusr/question_only/question_only_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,955 copying lm_eval/tasks/mmlusr/question_only/question_only_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,957 copying lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,959 copying lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,961 copying lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,963 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,965 copying lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,968 copying lm_eval/tasks/mmlusr/question_only/question_only_virology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,970 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,972 copying lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,974 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,976 copying lm_eval/tasks/mmlusr/question_only/question_only_prehistory.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,979 copying lm_eval/tasks/mmlusr/question_only/question_only_philosophy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,981 copying lm_eval/tasks/mmlusr/question_only/question_only_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,983 copying lm_eval/tasks/mmlusr/question_only/_question_only.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,985 copying lm_eval/tasks/mmlusr/question_only/question_only_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:18,987 copying lm_eval/tasks/mmlusr/answer_only/answer_only_global_facts.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:18,989 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_law.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:18,992 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:18,994 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:18,996 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:18,998 copying lm_eval/tasks/mmlusr/answer_only/answer_only_marketing.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,001 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,003 copying lm_eval/tasks/mmlusr/answer_only/answer_only_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,005 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,008 copying lm_eval/tasks/mmlusr/answer_only/answer_only_public_relations.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,010 copying lm_eval/tasks/mmlusr/answer_only/answer_only_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,013 copying lm_eval/tasks/mmlusr/answer_only/answer_only_human_aging.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,015 copying lm_eval/tasks/mmlusr/answer_only/answer_only_philosophy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,018 copying lm_eval/tasks/mmlusr/answer_only/answer_only_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,020 copying lm_eval/tasks/mmlusr/answer_only/answer_only_security_studies.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,023 copying lm_eval/tasks/mmlusr/answer_only/answer_only_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,025 copying lm_eval/tasks/mmlusr/answer_only/answer_only_astronomy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,027 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,029 copying lm_eval/tasks/mmlusr/answer_only/answer_only_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,031 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,034 copying lm_eval/tasks/mmlusr/answer_only/answer_only_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,036 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,038 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,040 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,042 copying lm_eval/tasks/mmlusr/answer_only/answer_only_international_law.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,044 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,046 copying lm_eval/tasks/mmlusr/answer_only/answer_only_virology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,048 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,051 copying lm_eval/tasks/mmlusr/answer_only/answer_only_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,053 copying lm_eval/tasks/mmlusr/answer_only/answer_only_anatomy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,055 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,057 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,059 copying lm_eval/tasks/mmlusr/answer_only/answer_only_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,061 copying lm_eval/tasks/mmlusr/answer_only/answer_only_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,064 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,066 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,068 copying lm_eval/tasks/mmlusr/answer_only/answer_only_nutrition.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,070 copying lm_eval/tasks/mmlusr/answer_only/answer_only_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,073 copying lm_eval/tasks/mmlusr/answer_only/answer_only_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,075 copying lm_eval/tasks/mmlusr/answer_only/_answer_only.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,077 copying lm_eval/tasks/mmlusr/answer_only/answer_only_sociology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,079 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,081 copying lm_eval/tasks/mmlusr/answer_only/answer_only_prehistory.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,083 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,085 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,087 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,089 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,092 copying lm_eval/tasks/mmlusr/answer_only/answer_only_management.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,094 copying lm_eval/tasks/mmlusr/answer_only/answer_only_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,096 copying lm_eval/tasks/mmlusr/answer_only/answer_only_econometrics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,098 copying lm_eval/tasks/mmlusr/answer_only/answer_only_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,100 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,102 copying lm_eval/tasks/mmlusr/answer_only/answer_only_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,104 copying lm_eval/tasks/mmlusr/answer_only/answer_only_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,106 copying lm_eval/tasks/mmlusr/answer_only/answer_only_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,109 copying lm_eval/tasks/mmlusr/answer_only/answer_only_world_religions.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,111 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,113 copying lm_eval/tasks/mmlusr/answer_only/answer_only_computer_security.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:19,115 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,118 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_econometrics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,120 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_anatomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,122 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,124 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,127 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_astronomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,129 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,131 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,133 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,135 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,138 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,140 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,142 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,144 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,146 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,148 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,150 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,152 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,154 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_sociology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,156 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,159 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_prehistory.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,161 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,163 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,166 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,168 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,170 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,172 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,175 copying lm_eval/tasks/mmlusr/question_and_answer/_question_and_answer.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,177 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,179 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,181 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,183 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_marketing.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,185 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,188 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,190 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,192 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_management.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,194 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_philosophy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,196 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_nutrition.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,199 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_aging.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,201 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,203 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,205 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,208 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_global_facts.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,210 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_security_studies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,212 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,214 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,217 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,219 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_computer_security.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,221 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_international_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,223 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,225 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,228 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,230 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,232 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,235 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,237 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,239 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_public_relations.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,241 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:19,243 copying lm_eval/tasks/egyhellaswag/egyhellaswag.yaml -> build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T14:29:19,245 copying lm_eval/tasks/wikitext/wikitext.yaml -> build/lib/lm_eval/tasks/wikitext 2026-05-11T14:29:19,247 copying lm_eval/tasks/mmlu_pro/mmlu_pro_health.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,249 copying lm_eval/tasks/mmlu_pro/mmlu_pro_biology.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,251 copying lm_eval/tasks/mmlu_pro/mmlu_pro_history.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,253 copying lm_eval/tasks/mmlu_pro/mmlu_pro_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,256 copying lm_eval/tasks/mmlu_pro/mmlu_pro_law.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,258 copying lm_eval/tasks/mmlu_pro/_mmlu_pro.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,260 copying lm_eval/tasks/mmlu_pro/mmlu_pro_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,262 copying lm_eval/tasks/mmlu_pro/mmlu_pro_math.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,264 copying lm_eval/tasks/mmlu_pro/mmlu_pro_other.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,267 copying lm_eval/tasks/mmlu_pro/mmlu_pro_economics.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,269 copying lm_eval/tasks/mmlu_pro/mmlu_pro_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,272 copying lm_eval/tasks/mmlu_pro/mmlu_pro_business.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,274 copying lm_eval/tasks/mmlu_pro/mmlu_pro_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,277 copying lm_eval/tasks/mmlu_pro/mmlu_pro_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,279 copying lm_eval/tasks/mmlu_pro/mmlu_pro_physics.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:19,281 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,283 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,286 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,288 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,290 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,292 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,294 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,296 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,298 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,300 copying lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,303 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,305 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,308 copying lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,310 copying lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,312 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,314 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,316 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,319 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,321 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,323 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,325 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,327 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,329 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,331 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,333 copying lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,336 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,338 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,340 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,342 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,344 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,346 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,348 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,350 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,352 copying lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,355 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,357 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,359 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,361 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,363 copying lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,365 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,368 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,370 copying lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,373 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,377 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,379 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,382 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,384 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,386 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,388 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,390 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,392 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,394 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,396 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,400 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,402 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,405 copying lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,407 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,409 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,411 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,414 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,416 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,418 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:19,423 copying lm_eval/tasks/README.md -> build/lib/lm_eval/tasks 2026-05-11T14:29:19,430 copying lm_eval/tasks/glue/README.md -> build/lib/lm_eval/tasks/glue 2026-05-11T14:29:19,434 copying lm_eval/tasks/nerel-bench/README.md -> build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:29:19,436 copying lm_eval/tasks/nerel-bench/_template_yaml -> build/lib/lm_eval/tasks/nerel-bench 2026-05-11T14:29:19,440 copying lm_eval/tasks/xnli_eu/xnli_common_yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T14:29:19,442 copying lm_eval/tasks/xnli_eu/README.md -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T14:29:19,445 copying lm_eval/tasks/winogrande/README.md -> build/lib/lm_eval/tasks/winogrande 2026-05-11T14:29:19,448 copying lm_eval/tasks/xstorycloze/README.md -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T14:29:19,451 copying lm_eval/tasks/haerae/README.md -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:29:19,453 copying lm_eval/tasks/haerae/_default_haerae_yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T14:29:19,455 copying lm_eval/tasks/humaneval/README.md -> build/lib/lm_eval/tasks/humaneval 2026-05-11T14:29:19,459 copying lm_eval/tasks/eus_proficiency/README.md -> build/lib/lm_eval/tasks/eus_proficiency 2026-05-11T14:29:19,461 copying lm_eval/tasks/swag/README.md -> build/lib/lm_eval/tasks/swag 2026-05-11T14:29:19,466 copying lm_eval/tasks/scrolls/README.md -> build/lib/lm_eval/tasks/scrolls 2026-05-11T14:29:19,469 copying lm_eval/tasks/simple_cooccurrence_bias/README.md -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:29:19,474 copying lm_eval/tasks/multiblimp/README.md -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:29:19,477 copying lm_eval/tasks/multiblimp/_template_yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T14:29:19,480 copying lm_eval/tasks/glianorex/README.md -> build/lib/lm_eval/tasks/glianorex 2026-05-11T14:29:19,482 copying lm_eval/tasks/discrim_eval/README.md -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T14:29:19,486 copying lm_eval/tasks/qa4mre/README.md -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T14:29:19,489 copying lm_eval/tasks/race/README.md -> build/lib/lm_eval/tasks/race 2026-05-11T14:29:19,491 copying lm_eval/tasks/bear/README.md -> build/lib/lm_eval/tasks/bear 2026-05-11T14:29:19,493 copying lm_eval/tasks/winogender/README.md -> build/lib/lm_eval/tasks/winogender 2026-05-11T14:29:19,496 copying lm_eval/tasks/bertaqa/README.md -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:29:19,499 copying lm_eval/tasks/bertaqa/_bertaqa_template -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T14:29:19,502 copying lm_eval/tasks/arab_culture/README.md -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:29:19,505 copying lm_eval/tasks/arab_culture/_default_arab_culture_mcq_template_yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T14:29:19,507 copying lm_eval/tasks/eus_trivia/README.md -> build/lib/lm_eval/tasks/eus_trivia 2026-05-11T14:29:19,510 copying lm_eval/tasks/arabic_leaderboard_complete/README.md -> build/lib/lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T14:29:19,525 copying lm_eval/tasks/acpbench/README.md -> build/lib/lm_eval/tasks/acpbench 2026-05-11T14:29:19,529 copying lm_eval/tasks/acpbench/gen_2shot/_gen_yaml_2shot -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:19,531 copying lm_eval/tasks/acpbench/gen_2shot/acp_grammar.lark -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:19,534 copying lm_eval/tasks/acpbench/mcq_cot_2shot/_mcq_cot_2shot_yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:19,536 copying lm_eval/tasks/acpbench/boolq_cot_2shot/_boolq_cot_2shot_yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:19,539 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/_gen_yaml_2shot -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:19,541 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_grammar.lark -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:19,544 copying lm_eval/tasks/jsonschema_bench/README.md -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T14:29:19,547 copying lm_eval/tasks/eus_reading/README.md -> build/lib/lm_eval/tasks/eus_reading 2026-05-11T14:29:19,550 copying lm_eval/tasks/aclue/_default_template_yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:29:19,552 copying lm_eval/tasks/aclue/README.md -> build/lib/lm_eval/tasks/aclue 2026-05-11T14:29:19,555 copying lm_eval/tasks/careqa/README.md -> build/lib/lm_eval/tasks/careqa 2026-05-11T14:29:19,559 copying lm_eval/tasks/icelandic_winogrande/README.md -> build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T14:29:19,563 copying lm_eval/tasks/hrm8k/README.md -> build/lib/lm_eval/tasks/hrm8k 2026-05-11T14:29:19,567 copying lm_eval/tasks/hrm8k/en/_hrm8k_en_yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T14:29:19,570 copying lm_eval/tasks/hrm8k/default/_hrm8k_yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T14:29:19,573 copying lm_eval/tasks/copal_id/README.md -> build/lib/lm_eval/tasks/copal_id 2026-05-11T14:29:19,578 copying lm_eval/tasks/arab_culture_completion/_default_arab_culture_completion_template_yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:19,581 copying lm_eval/tasks/arab_culture_completion/README.md -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:19,585 copying lm_eval/tasks/arithmetic/README.md -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T14:29:19,587 copying lm_eval/tasks/medtext/README.md -> build/lib/lm_eval/tasks/medtext 2026-05-11T14:29:19,591 copying lm_eval/tasks/pisa/README.md -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:29:19,594 copying lm_eval/tasks/pisa/_template_yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T14:29:19,597 copying lm_eval/tasks/xnli/xnli_common_yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:29:19,599 copying lm_eval/tasks/xnli/README.md -> build/lib/lm_eval/tasks/xnli 2026-05-11T14:29:19,603 copying lm_eval/tasks/openai-mmmlu/README.md -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:19,605 copying lm_eval/tasks/openai-mmmlu/subjects.json -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:19,608 copying lm_eval/tasks/openai-mmmlu/languages.json -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:19,610 copying lm_eval/tasks/openai-mmmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:19,613 copying lm_eval/tasks/meqsum/README.md -> build/lib/lm_eval/tasks/meqsum 2026-05-11T14:29:19,616 copying lm_eval/tasks/storycloze/README.md -> build/lib/lm_eval/tasks/storycloze 2026-05-11T14:29:19,619 copying lm_eval/tasks/global_mmlu/README.md -> build/lib/lm_eval/tasks/global_mmlu 2026-05-11T14:29:19,622 copying lm_eval/tasks/global_mmlu/full/de/_de_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:19,625 copying lm_eval/tasks/global_mmlu/full/hi/_hi_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:19,627 copying lm_eval/tasks/global_mmlu/full/bn/_bn_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:19,631 copying lm_eval/tasks/global_mmlu/full/ar/_ar_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:19,634 copying lm_eval/tasks/global_mmlu/full/es/_es_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:19,637 copying lm_eval/tasks/global_mmlu/full/pt/_pt_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:19,640 copying lm_eval/tasks/global_mmlu/full/fil/_fil_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:19,643 copying lm_eval/tasks/global_mmlu/full/lt/_lt_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:19,645 copying lm_eval/tasks/global_mmlu/full/cs/_cs_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:19,649 copying lm_eval/tasks/global_mmlu/full/ro/_ro_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:19,652 copying lm_eval/tasks/global_mmlu/full/ru/_ru_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:19,655 copying lm_eval/tasks/global_mmlu/full/sr/_sr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:19,658 copying lm_eval/tasks/global_mmlu/full/el/_el_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:19,661 copying lm_eval/tasks/global_mmlu/full/zh/_zh_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:19,664 copying lm_eval/tasks/global_mmlu/full/mg/_mg_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:19,668 copying lm_eval/tasks/global_mmlu/full/id/_id_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:19,671 copying lm_eval/tasks/global_mmlu/full/ha/_ha_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:19,674 copying lm_eval/tasks/global_mmlu/full/ko/_ko_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:19,677 copying lm_eval/tasks/global_mmlu/full/ja/_ja_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:19,680 copying lm_eval/tasks/global_mmlu/full/pl/_pl_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:19,684 copying lm_eval/tasks/global_mmlu/full/uk/_uk_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:19,686 copying lm_eval/tasks/global_mmlu/full/ig/_ig_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:19,689 copying lm_eval/tasks/global_mmlu/full/sw/_sw_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:19,692 copying lm_eval/tasks/global_mmlu/full/sn/_sn_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:19,695 copying lm_eval/tasks/global_mmlu/full/fr/_fr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:19,699 copying lm_eval/tasks/global_mmlu/full/en/_en_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:19,701 copying lm_eval/tasks/global_mmlu/full/ne/_ne_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:19,705 copying lm_eval/tasks/global_mmlu/full/am/_am_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:19,709 copying lm_eval/tasks/global_mmlu/full/yo/_yo_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:19,711 copying lm_eval/tasks/global_mmlu/full/te/_te_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:19,714 copying lm_eval/tasks/global_mmlu/full/ms/_ms_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:19,717 copying lm_eval/tasks/global_mmlu/full/he/_he_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:19,720 copying lm_eval/tasks/global_mmlu/full/sv/_sv_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:19,722 copying lm_eval/tasks/global_mmlu/full/vi/_vi_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:19,726 copying lm_eval/tasks/global_mmlu/full/ky/_ky_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:19,729 copying lm_eval/tasks/global_mmlu/full/fa/_fa_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:19,732 copying lm_eval/tasks/global_mmlu/full/nl/_nl_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:19,735 copying lm_eval/tasks/global_mmlu/full/ny/_ny_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:19,738 copying lm_eval/tasks/global_mmlu/full/tr/_tr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:19,741 copying lm_eval/tasks/global_mmlu/full/it/_it_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:19,744 copying lm_eval/tasks/global_mmlu/full/so/_so_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:19,747 copying lm_eval/tasks/global_mmlu/full/si/_si_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:19,750 copying lm_eval/tasks/global_mmlu/default/de/_de_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:19,754 copying lm_eval/tasks/global_mmlu/default/hi/_hi_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:19,756 copying lm_eval/tasks/global_mmlu/default/bn/_bn_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:19,760 copying lm_eval/tasks/global_mmlu/default/ar/_ar_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:19,762 copying lm_eval/tasks/global_mmlu/default/es/_es_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:19,765 copying lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:19,768 copying lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:19,772 copying lm_eval/tasks/global_mmlu/default/id/_id_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:19,774 copying lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:19,777 copying lm_eval/tasks/global_mmlu/default/ja/_ja_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:19,781 copying lm_eval/tasks/global_mmlu/default/sw/_sw_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:19,784 copying lm_eval/tasks/global_mmlu/default/fr/_fr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:19,788 copying lm_eval/tasks/global_mmlu/default/en/_en_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:19,791 copying lm_eval/tasks/global_mmlu/default/yo/_yo_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:19,794 copying lm_eval/tasks/global_mmlu/default/it/_it_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:19,796 copying lm_eval/tasks/mlqa/README.md -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:29:19,799 copying lm_eval/tasks/mlqa/mlqa_common_yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T14:29:19,802 copying lm_eval/tasks/japanese_leaderboard/requirements.txt -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:19,806 copying lm_eval/tasks/japanese_leaderboard/README.md -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:19,810 copying lm_eval/tasks/gpqa/README.md -> build/lib/lm_eval/tasks/gpqa 2026-05-11T14:29:19,812 copying lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T14:29:19,816 copying lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:19,820 copying lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:19,824 copying lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:19,827 copying lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:19,831 copying lm_eval/tasks/portuguese_bench/README.md -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T14:29:19,833 copying lm_eval/tasks/portuguese_bench/flores_pt/_flores_common_yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:19,839 copying lm_eval/tasks/ruler/README.md -> build/lib/lm_eval/tasks/ruler 2026-05-11T14:29:19,845 copying lm_eval/tasks/mmlu_prox/README.md -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T14:29:19,849 copying lm_eval/tasks/mmlu_prox/de/_de_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:19,851 copying lm_eval/tasks/mmlu_prox/de/_de_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:19,854 copying lm_eval/tasks/mmlu_prox/hi/_hi_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:19,857 copying lm_eval/tasks/mmlu_prox/hi/_hi_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:19,859 copying lm_eval/tasks/mmlu_prox/ur/_ur_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:19,861 copying lm_eval/tasks/mmlu_prox/ur/_ur_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:19,864 copying lm_eval/tasks/mmlu_prox/bn/_bn_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:19,867 copying lm_eval/tasks/mmlu_prox/bn/_bn_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:19,869 copying lm_eval/tasks/mmlu_prox/ar/_ar_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:19,872 copying lm_eval/tasks/mmlu_prox/ar/_ar_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:19,875 copying lm_eval/tasks/mmlu_prox/es/_es_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:19,877 copying lm_eval/tasks/mmlu_prox/es/_es_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:19,880 copying lm_eval/tasks/mmlu_prox/pt/_pt_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:19,882 copying lm_eval/tasks/mmlu_prox/pt/_pt_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:19,885 copying lm_eval/tasks/mmlu_prox/zu/_zu_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:19,888 copying lm_eval/tasks/mmlu_prox/zu/_zu_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:19,890 copying lm_eval/tasks/mmlu_prox/cs/_cs_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:19,893 copying lm_eval/tasks/mmlu_prox/cs/_cs_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:19,895 copying lm_eval/tasks/mmlu_prox/mr/_mr_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:19,898 copying lm_eval/tasks/mmlu_prox/mr/_mr_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:19,901 copying lm_eval/tasks/mmlu_prox/af/_af_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:19,903 copying lm_eval/tasks/mmlu_prox/af/_af_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:19,907 copying lm_eval/tasks/mmlu_prox/wo/_wo_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:19,909 copying lm_eval/tasks/mmlu_prox/wo/_wo_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:19,912 copying lm_eval/tasks/mmlu_prox/th/_th_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:19,915 copying lm_eval/tasks/mmlu_prox/th/_th_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:19,918 copying lm_eval/tasks/mmlu_prox/ru/_ru_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:19,920 copying lm_eval/tasks/mmlu_prox/ru/_ru_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:19,923 copying lm_eval/tasks/mmlu_prox/sr/_sr_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:19,925 copying lm_eval/tasks/mmlu_prox/sr/_sr_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:19,928 copying lm_eval/tasks/mmlu_prox/zh/_zh_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:19,930 copying lm_eval/tasks/mmlu_prox/zh/_zh_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:19,933 copying lm_eval/tasks/mmlu_prox/id/_id_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:19,936 copying lm_eval/tasks/mmlu_prox/id/_id_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:19,938 copying lm_eval/tasks/mmlu_prox/ko/_ko_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:19,940 copying lm_eval/tasks/mmlu_prox/ko/_ko_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:19,943 copying lm_eval/tasks/mmlu_prox/ja/_ja_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:19,945 copying lm_eval/tasks/mmlu_prox/ja/_ja_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:19,948 copying lm_eval/tasks/mmlu_prox/uk/_uk_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:19,951 copying lm_eval/tasks/mmlu_prox/uk/_uk_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:19,953 copying lm_eval/tasks/mmlu_prox/template/_lang_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/template 2026-05-11T14:29:19,956 copying lm_eval/tasks/mmlu_prox/sw/_sw_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:19,959 copying lm_eval/tasks/mmlu_prox/sw/_sw_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:19,961 copying lm_eval/tasks/mmlu_prox/fr/_fr_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:19,964 copying lm_eval/tasks/mmlu_prox/fr/_fr_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:19,967 copying lm_eval/tasks/mmlu_prox/en/_en_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:19,969 copying lm_eval/tasks/mmlu_prox/en/_en_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:19,972 copying lm_eval/tasks/mmlu_prox/ne/_ne_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:19,974 copying lm_eval/tasks/mmlu_prox/ne/_ne_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:19,977 copying lm_eval/tasks/mmlu_prox/yo/_yo_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:19,979 copying lm_eval/tasks/mmlu_prox/yo/_yo_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:19,982 copying lm_eval/tasks/mmlu_prox/te/_te_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:19,984 copying lm_eval/tasks/mmlu_prox/te/_te_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:19,986 copying lm_eval/tasks/mmlu_prox/vi/_vi_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:19,989 copying lm_eval/tasks/mmlu_prox/vi/_vi_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:19,991 copying lm_eval/tasks/mmlu_prox/hu/_hu_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:19,994 copying lm_eval/tasks/mmlu_prox/hu/_hu_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:19,996 copying lm_eval/tasks/mmlu_prox/it/_it_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:19,999 copying lm_eval/tasks/mmlu_prox/it/_it_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:20,002 copying lm_eval/tasks/swde/README.md -> build/lib/lm_eval/tasks/swde 2026-05-11T14:29:20,004 copying lm_eval/tasks/kmmlu/README.md -> build/lib/lm_eval/tasks/kmmlu 2026-05-11T14:29:20,007 copying lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:20,009 copying lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:20,011 copying lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:20,013 copying lm_eval/tasks/kmmlu/default/_default_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:20,015 copying lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:20,018 copying lm_eval/tasks/arabicmmlu/README.md -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:20,020 copying lm_eval/tasks/arabicmmlu/_default_arabicmmlu_template_yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:20,023 copying lm_eval/tasks/noticia/README.md -> build/lib/lm_eval/tasks/noticia 2026-05-11T14:29:20,027 copying lm_eval/tasks/bigbench/multiple_choice_template_a_yaml -> build/lib/lm_eval/tasks/bigbench 2026-05-11T14:29:20,029 copying lm_eval/tasks/bigbench/README.md -> build/lib/lm_eval/tasks/bigbench 2026-05-11T14:29:20,032 copying lm_eval/tasks/bigbench/multiple_choice_template_b_yaml -> build/lib/lm_eval/tasks/bigbench 2026-05-11T14:29:20,034 copying lm_eval/tasks/bigbench/generate_until_template_yaml -> build/lib/lm_eval/tasks/bigbench 2026-05-11T14:29:20,038 copying lm_eval/tasks/global_piqa/README.md -> build/lib/lm_eval/tasks/global_piqa 2026-05-11T14:29:20,040 copying lm_eval/tasks/global_piqa/completions/_template -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:20,043 copying lm_eval/tasks/global_piqa/prompted/_template -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:20,046 copying lm_eval/tasks/bbh/README.md -> build/lib/lm_eval/tasks/bbh 2026-05-11T14:29:20,049 copying lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:20,051 copying lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:20,054 copying lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:20,056 copying lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:20,059 copying lm_eval/tasks/babi/README.md -> build/lib/lm_eval/tasks/babi 2026-05-11T14:29:20,062 copying lm_eval/tasks/drop/README.md -> build/lib/lm_eval/tasks/drop 2026-05-11T14:29:20,065 copying lm_eval/tasks/toxigen/README.md -> build/lib/lm_eval/tasks/toxigen 2026-05-11T14:29:20,068 copying lm_eval/tasks/pubmedqa/README.md -> build/lib/lm_eval/tasks/pubmedqa 2026-05-11T14:29:20,071 copying lm_eval/tasks/anli/README.md -> build/lib/lm_eval/tasks/anli 2026-05-11T14:29:20,074 copying lm_eval/tasks/infinitebench/_infinitebench_common_yaml -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:20,076 copying lm_eval/tasks/infinitebench/README.md -> build/lib/lm_eval/tasks/infinitebench 2026-05-11T14:29:20,079 copying lm_eval/tasks/agieval/README.md -> build/lib/lm_eval/tasks/agieval 2026-05-11T14:29:20,082 copying lm_eval/tasks/prost/README.md -> build/lib/lm_eval/tasks/prost 2026-05-11T14:29:20,085 copying lm_eval/tasks/unitxt/unitxt -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:20,087 copying lm_eval/tasks/unitxt/unitxt_multimodal -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:20,089 copying lm_eval/tasks/unitxt/README.md -> build/lib/lm_eval/tasks/unitxt 2026-05-11T14:29:20,091 creating build/lib/lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T14:29:20,092 copying lm_eval/tasks/model_written_evals/winogenerated/winogenerated -> build/lib/lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T14:29:20,095 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:20,098 copying lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:20,101 copying lm_eval/tasks/metabench/README.md -> build/lib/lm_eval/tasks/metabench 2026-05-11T14:29:20,104 copying lm_eval/tasks/cmmlu/_default_template_yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:20,107 copying lm_eval/tasks/cmmlu/README.md -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T14:29:20,111 copying lm_eval/tasks/paws-x/pawsx_template_yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:20,113 copying lm_eval/tasks/paws-x/README.md -> build/lib/lm_eval/tasks/paws-x 2026-05-11T14:29:20,117 copying lm_eval/tasks/include/README.md -> build/lib/lm_eval/tasks/include 2026-05-11T14:29:20,120 copying lm_eval/tasks/include/few_shot_en/Indonesian/_indonesian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:20,123 copying lm_eval/tasks/include/few_shot_en/Tagalog/_tagalog_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:20,126 copying lm_eval/tasks/include/few_shot_en/North Macedonian/_north macedonian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:20,129 copying lm_eval/tasks/include/few_shot_en/Malayalam/_malayalam_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:20,133 copying lm_eval/tasks/include/few_shot_en/Belarusian/_belarusian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:20,135 copying lm_eval/tasks/include/few_shot_en/Kazakh/_kazakh_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:20,139 copying lm_eval/tasks/include/few_shot_en/Persian/_persian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:20,142 copying lm_eval/tasks/include/few_shot_en/Arabic/_arabic_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:20,146 copying lm_eval/tasks/include/few_shot_en/Ukrainian/_ukrainian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:20,149 copying lm_eval/tasks/include/few_shot_en/Telugu/_telugu_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:20,152 copying lm_eval/tasks/include/few_shot_en/Urdu/_urdu_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:20,156 copying lm_eval/tasks/include/few_shot_en/Lithuanian/_lithuanian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:20,159 copying lm_eval/tasks/include/few_shot_en/Vietnamese/_vietnamese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:20,163 copying lm_eval/tasks/include/few_shot_en/Albanian/_albanian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:20,166 copying lm_eval/tasks/include/few_shot_en/Finnish/_finnish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:20,169 copying lm_eval/tasks/include/few_shot_en/Spanish/_spanish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:20,171 copying lm_eval/tasks/include/few_shot_en/Nepali/_nepali_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:20,175 copying lm_eval/tasks/include/few_shot_en/Polish/_polish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:20,177 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/_azerbaijani_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:20,180 copying lm_eval/tasks/include/few_shot_en/Serbian/_serbian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:20,184 copying lm_eval/tasks/include/few_shot_en/Portuguese/_portuguese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:20,187 copying lm_eval/tasks/include/few_shot_en/French/_french_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:20,190 copying lm_eval/tasks/include/few_shot_en/Uzbek/_uzbek_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:20,193 copying lm_eval/tasks/include/few_shot_en/Malay/_malay_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:20,196 copying lm_eval/tasks/include/few_shot_en/Italian/_italian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:20,199 copying lm_eval/tasks/include/few_shot_en/Estonian/_estonian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:20,202 copying lm_eval/tasks/include/few_shot_en/Bengali/_bengali_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:20,205 copying lm_eval/tasks/include/few_shot_en/Georgian/_georgian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:20,208 copying lm_eval/tasks/include/few_shot_en/Russian/_russian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:20,211 copying lm_eval/tasks/include/few_shot_en/Korean/_korean_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:20,214 copying lm_eval/tasks/include/few_shot_en/Chinese/_chinese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:20,217 copying lm_eval/tasks/include/few_shot_en/Croatian/_croatian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:20,221 copying lm_eval/tasks/include/few_shot_en/Dutch/_dutch_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:20,224 copying lm_eval/tasks/include/few_shot_en/Hindi/_hindi_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:20,227 copying lm_eval/tasks/include/few_shot_en/Tamil/_tamil_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:20,230 copying lm_eval/tasks/include/few_shot_en/Hungarian/_hungarian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:20,233 copying lm_eval/tasks/include/few_shot_en/Turkish/_turkish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:20,236 copying lm_eval/tasks/include/few_shot_en/Greek/_greek_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:20,240 copying lm_eval/tasks/include/few_shot_en/Armenian/_armenian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:20,242 copying lm_eval/tasks/include/few_shot_en/Hebrew/_hebrew_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:20,245 copying lm_eval/tasks/include/few_shot_en/German/_german_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:20,248 copying lm_eval/tasks/include/few_shot_en/Japanese/_japanese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:20,251 copying lm_eval/tasks/include/few_shot_en/Basque/_basque_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:20,254 copying lm_eval/tasks/include/few_shot_en/Bulgarian/_bulgarian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:20,257 copying lm_eval/tasks/include/default/Indonesian/_indonesian_template_yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:20,260 copying lm_eval/tasks/include/default/Tagalog/_tagalog_template_yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:20,263 copying lm_eval/tasks/include/default/North Macedonian/_north macedonian_template_yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:20,266 copying lm_eval/tasks/include/default/Malayalam/_malayalam_template_yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:20,269 copying lm_eval/tasks/include/default/Belarusian/_belarusian_template_yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:20,272 copying lm_eval/tasks/include/default/Kazakh/_kazakh_template_yaml -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:20,275 copying lm_eval/tasks/include/default/Persian/_persian_template_yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:20,278 copying lm_eval/tasks/include/default/Arabic/_arabic_template_yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:20,281 copying lm_eval/tasks/include/default/Ukrainian/_ukrainian_template_yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:20,285 copying lm_eval/tasks/include/default/Telugu/_telugu_template_yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:20,287 copying lm_eval/tasks/include/default/Urdu/_urdu_template_yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:20,290 copying lm_eval/tasks/include/default/Lithuanian/_lithuanian_template_yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:20,293 copying lm_eval/tasks/include/default/Vietnamese/_vietnamese_template_yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:20,296 copying lm_eval/tasks/include/default/Albanian/_albanian_template_yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:20,299 copying lm_eval/tasks/include/default/Finnish/_finnish_template_yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:20,302 copying lm_eval/tasks/include/default/Spanish/_spanish_template_yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:20,306 copying lm_eval/tasks/include/default/Nepali/_nepali_template_yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:20,309 copying lm_eval/tasks/include/default/Polish/_polish_template_yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T14:29:20,312 copying lm_eval/tasks/include/default/Azerbaijani/_azerbaijani_template_yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:20,315 copying lm_eval/tasks/include/default/Serbian/_serbian_template_yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:20,317 copying lm_eval/tasks/include/default/Portuguese/_portuguese_template_yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:20,320 copying lm_eval/tasks/include/default/French/_french_template_yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T14:29:20,324 copying lm_eval/tasks/include/default/Uzbek/_uzbek_template_yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:20,326 copying lm_eval/tasks/include/default/Malay/_malay_template_yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T14:29:20,329 copying lm_eval/tasks/include/default/Italian/_italian_template_yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:20,332 copying lm_eval/tasks/include/default/Estonian/_estonian_template_yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:20,335 copying lm_eval/tasks/include/default/Bengali/_bengali_template_yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:20,339 copying lm_eval/tasks/include/default/Georgian/_georgian_template_yaml -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:20,341 copying lm_eval/tasks/include/default/Russian/_russian_template_yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:20,344 copying lm_eval/tasks/include/default/Korean/_korean_template_yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T14:29:20,347 copying lm_eval/tasks/include/default/Chinese/_chinese_template_yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:20,350 copying lm_eval/tasks/include/default/Croatian/_croatian_template_yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:20,353 copying lm_eval/tasks/include/default/Dutch/_dutch_template_yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:20,356 copying lm_eval/tasks/include/default/Hindi/_hindi_template_yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:20,359 copying lm_eval/tasks/include/default/Tamil/_tamil_template_yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:20,361 copying lm_eval/tasks/include/default/Hungarian/_hungarian_template_yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:20,364 copying lm_eval/tasks/include/default/Turkish/_turkish_template_yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:20,367 copying lm_eval/tasks/include/default/Greek/_greek_template_yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:20,370 copying lm_eval/tasks/include/default/Armenian/_armenian_template_yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:20,373 copying lm_eval/tasks/include/default/Hebrew/_hebrew_template_yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:20,376 copying lm_eval/tasks/include/default/German/_german_template_yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T14:29:20,379 copying lm_eval/tasks/include/default/Japanese/_japanese_template_yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:20,382 copying lm_eval/tasks/include/default/Basque/_basque_template_yaml -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T14:29:20,386 copying lm_eval/tasks/include/default/Bulgarian/_bulgarian_template_yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:20,388 copying lm_eval/tasks/include/few_shot_og/Indonesian/_indonesian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:20,391 copying lm_eval/tasks/include/few_shot_og/Tagalog/_tagalog_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:20,394 copying lm_eval/tasks/include/few_shot_og/North Macedonian/_north macedonian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:20,397 copying lm_eval/tasks/include/few_shot_og/Malayalam/_malayalam_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:20,400 copying lm_eval/tasks/include/few_shot_og/Belarusian/_belarusian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:20,403 copying lm_eval/tasks/include/few_shot_og/Kazakh/_kazakh_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:20,406 copying lm_eval/tasks/include/few_shot_og/Persian/_persian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:20,409 copying lm_eval/tasks/include/few_shot_og/Arabic/_arabic_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:20,412 copying lm_eval/tasks/include/few_shot_og/Ukrainian/_ukrainian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:20,416 copying lm_eval/tasks/include/few_shot_og/Telugu/_telugu_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:20,418 copying lm_eval/tasks/include/few_shot_og/Urdu/_urdu_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:20,421 copying lm_eval/tasks/include/few_shot_og/Lithuanian/_lithuanian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:20,424 copying lm_eval/tasks/include/few_shot_og/Vietnamese/_vietnamese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:20,427 copying lm_eval/tasks/include/few_shot_og/Albanian/_albanian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:20,430 copying lm_eval/tasks/include/few_shot_og/Finnish/_finnish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:20,433 copying lm_eval/tasks/include/few_shot_og/Spanish/_spanish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:20,436 copying lm_eval/tasks/include/few_shot_og/Nepali/_nepali_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:20,439 copying lm_eval/tasks/include/few_shot_og/Polish/_polish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:20,442 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/_azerbaijani_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:20,445 copying lm_eval/tasks/include/few_shot_og/Serbian/_serbian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:20,449 copying lm_eval/tasks/include/few_shot_og/Portuguese/_portuguese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:20,452 copying lm_eval/tasks/include/few_shot_og/French/_french_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:20,454 copying lm_eval/tasks/include/few_shot_og/Uzbek/_uzbek_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:20,458 copying lm_eval/tasks/include/few_shot_og/Malay/_malay_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:20,461 copying lm_eval/tasks/include/few_shot_og/Italian/_italian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:20,464 copying lm_eval/tasks/include/few_shot_og/Estonian/_estonian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:20,467 copying lm_eval/tasks/include/few_shot_og/Bengali/_bengali_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:20,471 copying lm_eval/tasks/include/few_shot_og/Georgian/_georgian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:20,473 copying lm_eval/tasks/include/few_shot_og/Russian/_russian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:20,476 copying lm_eval/tasks/include/few_shot_og/Korean/_korean_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:20,479 copying lm_eval/tasks/include/few_shot_og/Chinese/_chinese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:20,482 copying lm_eval/tasks/include/few_shot_og/Croatian/_croatian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:20,485 copying lm_eval/tasks/include/few_shot_og/Dutch/_dutch_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:20,488 copying lm_eval/tasks/include/few_shot_og/Hindi/_hindi_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:20,491 copying lm_eval/tasks/include/few_shot_og/Tamil/_tamil_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:20,494 copying lm_eval/tasks/include/few_shot_og/Hungarian/_hungarian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:20,497 copying lm_eval/tasks/include/few_shot_og/Turkish/_turkish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:20,500 copying lm_eval/tasks/include/few_shot_og/Greek/_greek_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:20,503 copying lm_eval/tasks/include/few_shot_og/Armenian/_armenian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:20,506 copying lm_eval/tasks/include/few_shot_og/Hebrew/_hebrew_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:20,509 copying lm_eval/tasks/include/few_shot_og/German/_german_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:20,512 copying lm_eval/tasks/include/few_shot_og/Japanese/_japanese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:20,515 copying lm_eval/tasks/include/few_shot_og/Basque/_basque_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:20,518 copying lm_eval/tasks/include/few_shot_og/Bulgarian/_bulgarian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:20,521 copying lm_eval/tasks/blimp/README.md -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:20,524 copying lm_eval/tasks/blimp/_template_yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T14:29:20,526 copying lm_eval/tasks/mmmu/README.md -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:20,528 copying lm_eval/tasks/mmmu/_template_yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T14:29:20,532 copying lm_eval/tasks/tmlu/README.md -> build/lib/lm_eval/tasks/tmlu 2026-05-11T14:29:20,534 copying lm_eval/tasks/tmlu/subject.tsv -> build/lib/lm_eval/tasks/tmlu 2026-05-11T14:29:20,536 copying lm_eval/tasks/tmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T14:29:20,540 copying lm_eval/tasks/egymmlu/_default_egymmlu_template_yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:20,542 copying lm_eval/tasks/egymmlu/README.md -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T14:29:20,546 copying lm_eval/tasks/moral_stories/README.md -> build/lib/lm_eval/tasks/moral_stories 2026-05-11T14:29:20,549 copying lm_eval/tasks/mastermind/README.md -> build/lib/lm_eval/tasks/mastermind 2026-05-11T14:29:20,552 copying lm_eval/tasks/med_concepts_qa/_default_template_yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:20,554 copying lm_eval/tasks/med_concepts_qa/README.md -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:20,558 copying lm_eval/tasks/babilong/README.md -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:20,561 copying lm_eval/tasks/babilong/_babilong_common_yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T14:29:20,563 copying lm_eval/tasks/longbench2/_longbench_common_yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:20,565 copying lm_eval/tasks/longbench2/README.md -> build/lib/lm_eval/tasks/longbench2 2026-05-11T14:29:20,567 copying lm_eval/tasks/code_x_glue/code-text/_default_template_yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:20,570 copying lm_eval/tasks/code_x_glue/code-text/README.md -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:20,573 copying lm_eval/tasks/sciq/README.md -> build/lib/lm_eval/tasks/sciq 2026-05-11T14:29:20,578 copying lm_eval/tasks/tinyBenchmarks/README.md -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:20,580 copying lm_eval/tasks/chartqa/README.md -> build/lib/lm_eval/tasks/chartqa 2026-05-11T14:29:20,583 copying lm_eval/tasks/minerva_math/README.md -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T14:29:20,587 copying lm_eval/tasks/llama3/README.md -> build/lib/lm_eval/tasks/llama3 2026-05-11T14:29:20,589 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:20,592 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:20,595 copying lm_eval/tasks/llama3/instruct/mmlu_de/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:20,598 copying lm_eval/tasks/llama3/instruct/mmlu_es/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:20,601 copying lm_eval/tasks/llama3/instruct/mmlu_it/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:20,604 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:20,607 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:20,610 copying lm_eval/tasks/llama3/instruct/mmlu/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:20,612 copying lm_eval/tasks/llama3/instruct/mmlu_pro/_default_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:20,615 copying lm_eval/tasks/llama3/instruct/mmlu_th/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:20,618 copying lm_eval/tasks/commonsense_qa/README.md -> build/lib/lm_eval/tasks/commonsense_qa 2026-05-11T14:29:20,621 copying lm_eval/tasks/zhoblimp/README.md -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:20,623 copying lm_eval/tasks/zhoblimp/_template_yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T14:29:20,625 copying lm_eval/tasks/leaderboard/README.md -> build/lib/lm_eval/tasks/leaderboard 2026-05-11T14:29:20,628 copying lm_eval/tasks/leaderboard/musr/_template_yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:20,631 copying lm_eval/tasks/leaderboard/bbh_mc/_fewshot_template_yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:20,633 copying lm_eval/tasks/leaderboard/gpqa/_template_yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:20,639 copying lm_eval/tasks/leaderboard/math/_template_yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:20,643 copying lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:20,645 copying lm_eval/tasks/hendrycks_ethics/README.md -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:20,649 copying lm_eval/tasks/squad_completion/README.md -> build/lib/lm_eval/tasks/squad_completion 2026-05-11T14:29:20,651 copying lm_eval/tasks/triviaqa/README.md -> build/lib/lm_eval/tasks/triviaqa 2026-05-11T14:29:20,654 copying lm_eval/tasks/belebele/_default_template_yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:20,656 copying lm_eval/tasks/belebele/README.md -> build/lib/lm_eval/tasks/belebele 2026-05-11T14:29:20,659 copying lm_eval/tasks/basque_bench/README.md -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T14:29:20,662 copying lm_eval/tasks/basque_bench/flores_eu/_flores_common_yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:20,665 copying lm_eval/tasks/kobest/README.md -> build/lib/lm_eval/tasks/kobest 2026-05-11T14:29:20,668 copying lm_eval/tasks/inverse_scaling/_some_results -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:20,670 copying lm_eval/tasks/inverse_scaling/README.md -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:20,673 copying lm_eval/tasks/inverse_scaling/_inverse_scaling_mc_yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:20,675 copying lm_eval/tasks/logiqa/README.md -> build/lib/lm_eval/tasks/logiqa 2026-05-11T14:29:20,677 copying lm_eval/tasks/mutual/README.md -> build/lib/lm_eval/tasks/mutual 2026-05-11T14:29:20,680 copying lm_eval/tasks/galician_bench/README.md -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T14:29:20,683 copying lm_eval/tasks/galician_bench/flores_gl/_flores_common_yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:20,686 copying lm_eval/tasks/xcopa/README.md -> build/lib/lm_eval/tasks/xcopa 2026-05-11T14:29:20,689 copying lm_eval/tasks/cnn_dailymail/README.md -> build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T14:29:20,694 copying lm_eval/tasks/ifeval/README.md -> build/lib/lm_eval/tasks/ifeval 2026-05-11T14:29:20,703 copying lm_eval/tasks/aradice/README.md -> build/lib/lm_eval/tasks/aradice 2026-05-11T14:29:20,708 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/_default_template_yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:20,712 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/_default_template_yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:20,726 copying lm_eval/tasks/lambada_multilingual_stablelm/README.md -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:20,729 copying lm_eval/tasks/mediqa_qa2019/README.md -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:20,734 copying lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T14:29:20,738 copying lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:20,741 copying lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:20,743 copying lm_eval/tasks/okapi/arc_multilingual/_arc_yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:20,745 copying lm_eval/tasks/okapi/arc_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:20,748 copying lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:20,751 copying lm_eval/tasks/okapi/truthfulqa_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:20,754 copying lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:20,756 copying lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:20,758 copying lm_eval/tasks/graphwalks/README.md -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T14:29:20,761 copying lm_eval/tasks/fld/README.md -> build/lib/lm_eval/tasks/fld 2026-05-11T14:29:20,764 copying lm_eval/tasks/aexams/_default_template_yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:20,766 copying lm_eval/tasks/aexams/README.md -> build/lib/lm_eval/tasks/aexams 2026-05-11T14:29:20,769 copying lm_eval/tasks/afrixnli/README.md -> build/lib/lm_eval/tasks/afrixnli 2026-05-11T14:29:20,772 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:20,775 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:20,778 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:20,781 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:20,784 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:20,787 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:20,790 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:20,792 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:20,796 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:20,798 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:20,801 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:20,804 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:20,808 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:20,811 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:20,813 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:20,816 copying lm_eval/tasks/wmdp/_default_template_yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T14:29:20,818 copying lm_eval/tasks/wmdp/README.md -> build/lib/lm_eval/tasks/wmdp 2026-05-11T14:29:20,821 copying lm_eval/tasks/c4/README.md -> build/lib/lm_eval/tasks/c4 2026-05-11T14:29:20,824 copying lm_eval/tasks/bangla/README.md -> build/lib/lm_eval/tasks/bangla 2026-05-11T14:29:20,826 copying lm_eval/tasks/toksuite/README.md -> build/lib/lm_eval/tasks/toksuite 2026-05-11T14:29:20,831 copying lm_eval/tasks/jfinqa/README.md -> build/lib/lm_eval/tasks/jfinqa 2026-05-11T14:29:20,834 copying lm_eval/tasks/gsm8k_platinum/README.md -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:20,836 copying lm_eval/tasks/piqa/README.md -> build/lib/lm_eval/tasks/piqa 2026-05-11T14:29:20,839 copying lm_eval/tasks/wmt2016/README.md -> build/lib/lm_eval/tasks/wmt2016 2026-05-11T14:29:20,842 copying lm_eval/tasks/mela/README.md -> build/lib/lm_eval/tasks/mela 2026-05-11T14:29:20,844 copying lm_eval/tasks/ulqa/README.md -> build/lib/lm_eval/tasks/ulqa 2026-05-11T14:29:20,847 copying lm_eval/tasks/pile/README.md -> build/lib/lm_eval/tasks/pile 2026-05-11T14:29:20,849 copying lm_eval/tasks/ceval/_default_ceval_yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:20,851 copying lm_eval/tasks/ceval/README.md -> build/lib/lm_eval/tasks/ceval 2026-05-11T14:29:20,854 copying lm_eval/tasks/polemo2/README.md -> build/lib/lm_eval/tasks/polemo2 2026-05-11T14:29:20,857 copying lm_eval/tasks/coqa/README.md -> build/lib/lm_eval/tasks/coqa 2026-05-11T14:29:20,860 copying lm_eval/tasks/olaph/README.md -> build/lib/lm_eval/tasks/olaph 2026-05-11T14:29:20,864 copying lm_eval/tasks/click/README.md -> build/lib/lm_eval/tasks/click 2026-05-11T14:29:20,867 copying lm_eval/tasks/click/click_lang/_default_click_lang_yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T14:29:20,869 copying lm_eval/tasks/click/click_cul/_default_click_cul_yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T14:29:20,872 copying lm_eval/tasks/afrimmlu/fewshot.sh -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T14:29:20,874 copying lm_eval/tasks/afrimmlu/README.md -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T14:29:20,878 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:20,881 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:20,884 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:20,887 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:20,890 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:20,893 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:20,895 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:20,898 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:20,901 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:20,905 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:20,908 copying lm_eval/tasks/unscramble/README.md -> build/lib/lm_eval/tasks/unscramble 2026-05-11T14:29:20,910 copying lm_eval/tasks/mbpp/README.md -> build/lib/lm_eval/tasks/mbpp 2026-05-11T14:29:20,913 copying lm_eval/tasks/pile_10k/README.md -> build/lib/lm_eval/tasks/pile_10k 2026-05-11T14:29:20,916 copying lm_eval/tasks/nq_open/README.md -> build/lib/lm_eval/tasks/nq_open 2026-05-11T14:29:20,918 copying lm_eval/tasks/catalan_bench/_cabreu_common_yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:20,920 copying lm_eval/tasks/catalan_bench/_arc_ca_common_yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:20,923 copying lm_eval/tasks/catalan_bench/README.md -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T14:29:20,926 copying lm_eval/tasks/catalan_bench/flores_ca/_flores_common_yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:20,929 copying lm_eval/tasks/catalan_bench/phrases_va/_phrases_va_common -> build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:20,932 copying lm_eval/tasks/bbq/README.md -> build/lib/lm_eval/tasks/bbq 2026-05-11T14:29:20,936 copying lm_eval/tasks/humaneval_infilling/README.md -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:20,939 copying lm_eval/tasks/mc_taco/README.md -> build/lib/lm_eval/tasks/mc_taco 2026-05-11T14:29:20,942 copying lm_eval/tasks/mgsm/README.md -> build/lib/lm_eval/tasks/mgsm 2026-05-11T14:29:20,945 copying lm_eval/tasks/mgsm/gen_yaml.sh -> build/lib/lm_eval/tasks/mgsm 2026-05-11T14:29:20,948 copying lm_eval/tasks/mgsm/direct/direct_yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:20,950 copying lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:20,953 copying lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:20,955 copying lm_eval/tasks/super_glue/README.md -> build/lib/lm_eval/tasks/super_glue 2026-05-11T14:29:20,965 copying lm_eval/tasks/mimic_repsum/README.md -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T14:29:20,969 copying lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:20,971 copying lm_eval/tasks/xwinograd/README.md -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T14:29:20,976 copying lm_eval/tasks/qasper/README.md -> build/lib/lm_eval/tasks/qasper 2026-05-11T14:29:21,197 copying lm_eval/tasks/tmmluplus/README.md -> build/lib/lm_eval/tasks/tmmluplus 2026-05-11T14:29:21,199 copying lm_eval/tasks/tmmluplus/subject.tsv -> build/lib/lm_eval/tasks/tmmluplus 2026-05-11T14:29:21,203 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_template_yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:21,205 copying lm_eval/tasks/turkishmmlu/README.md -> build/lib/lm_eval/tasks/turkishmmlu 2026-05-11T14:29:21,208 copying lm_eval/tasks/turkishmmlu/config/_turkishmmlu_default_yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:21,211 copying lm_eval/tasks/turkishmmlu/config_cot/_turkishmmlu_cot_default_yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:21,213 copying lm_eval/tasks/afrimgsm/README.md -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T14:29:21,215 copying lm_eval/tasks/afrimgsm/run.sh -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T14:29:21,218 copying lm_eval/tasks/afrimgsm/gen_yaml.sh -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T14:29:21,221 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:21,223 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:21,225 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:21,227 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:21,230 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:21,232 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:21,235 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:21,237 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:21,240 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:21,242 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:21,245 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:21,248 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:21,250 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:21,252 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:21,255 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:21,257 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:21,259 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:21,262 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:21,265 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:21,267 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:21,270 copying lm_eval/tasks/openbookqa/README.md -> build/lib/lm_eval/tasks/openbookqa 2026-05-11T14:29:21,272 copying lm_eval/tasks/crows_pairs/README.md -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T14:29:21,276 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/README.md -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:21,279 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/_var5shots_template_yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:21,282 copying lm_eval/tasks/e2lmc/mmlu_early_training/README.md -> build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:29:21,284 copying lm_eval/tasks/e2lmc/noor/README.md -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:21,286 copying lm_eval/tasks/e2lmc/noor/_noor_template -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:21,289 copying lm_eval/tasks/headqa/README.md -> build/lib/lm_eval/tasks/headqa 2026-05-11T14:29:21,291 copying lm_eval/tasks/libra/README.md -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:21,293 copying lm_eval/tasks/libra/_template_yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T14:29:21,296 copying lm_eval/tasks/arabic_leaderboard_light/README.md -> build/lib/lm_eval/tasks/arabic_leaderboard_light 2026-05-11T14:29:21,311 copying lm_eval/tasks/slr_bench/slr_bench_common_yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:21,313 copying lm_eval/tasks/slr_bench/README.md -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T14:29:21,315 copying lm_eval/tasks/logiqa2/README.md -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T14:29:21,320 copying lm_eval/tasks/longbench/README.md -> build/lib/lm_eval/tasks/longbench 2026-05-11T14:29:21,323 copying lm_eval/tasks/asdiv/README.md -> build/lib/lm_eval/tasks/asdiv 2026-05-11T14:29:21,325 copying lm_eval/tasks/basqueglue/README.md -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T14:29:21,328 copying lm_eval/tasks/cruxeval/README.md -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:21,331 copying lm_eval/tasks/cruxeval/cruxeval_common_yaml -> build/lib/lm_eval/tasks/cruxeval 2026-05-11T14:29:21,333 copying lm_eval/tasks/mathqa/README.md -> build/lib/lm_eval/tasks/mathqa 2026-05-11T14:29:21,337 copying lm_eval/tasks/alghafa/copa_ar/README.md -> build/lib/lm_eval/tasks/alghafa/copa_ar 2026-05-11T14:29:21,340 copying lm_eval/tasks/alghafa/piqa_ar/README.md -> build/lib/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T14:29:21,343 copying lm_eval/tasks/mmlu-redux-spanish/README.md -> build/lib/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T14:29:21,347 copying lm_eval/tasks/mmlu-redux-spanish/generative/_default_template_spanish_yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:21,350 copying lm_eval/tasks/darija_bench/README.md -> build/lib/lm_eval/tasks/darija_bench 2026-05-11T14:29:21,353 copying lm_eval/tasks/darija_bench/darija_sentiment/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:21,355 copying lm_eval/tasks/darija_bench/darija_sentiment/default_darija_sentiment_template_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:21,358 copying lm_eval/tasks/darija_bench/darija_summarization/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:21,361 copying lm_eval/tasks/darija_bench/darija_summarization/summarization_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:21,364 copying lm_eval/tasks/darija_bench/darija_translation/translation_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:21,367 copying lm_eval/tasks/darija_bench/darija_translation/madar_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:21,370 copying lm_eval/tasks/darija_bench/darija_translation/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:21,373 copying lm_eval/tasks/darija_bench/darija_translation/doda_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:21,376 copying lm_eval/tasks/darija_bench/darija_translation/flores_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:21,380 copying lm_eval/tasks/darija_bench/darija_translation/seed_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:21,383 copying lm_eval/tasks/darija_bench/darija_transliteration/transliteration_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:21,385 copying lm_eval/tasks/darija_bench/darija_transliteration/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:21,389 copying lm_eval/tasks/siqa/README.md -> build/lib/lm_eval/tasks/siqa 2026-05-11T14:29:21,393 copying lm_eval/tasks/hellaswag/README.md -> build/lib/lm_eval/tasks/hellaswag 2026-05-11T14:29:21,397 copying lm_eval/tasks/lm_syneval/README.md -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:21,401 copying lm_eval/tasks/lm_syneval/_template_yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T14:29:21,405 copying lm_eval/tasks/blimp_nl/README.md -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:21,409 copying lm_eval/tasks/blimp_nl/_template_yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T14:29:21,412 copying lm_eval/tasks/afrobench/README.md -> build/lib/lm_eval/tasks/afrobench 2026-05-11T14:29:21,416 copying lm_eval/tasks/afrobench/naijarc/README.md -> build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T14:29:21,420 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:21,424 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:21,427 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:21,432 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:21,436 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:21,439 creating build/lib/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T14:29:21,441 copying lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench.sh -> build/lib/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T14:29:21,445 copying lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench_lite.sh -> build/lib/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T14:29:21,449 copying lm_eval/tasks/afrobench/ntrex/README.md -> build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T14:29:21,453 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:21,457 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:21,461 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:21,464 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:21,469 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:21,472 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:21,476 copying lm_eval/tasks/afrobench/afrisenti/fewshot.sh -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:21,480 copying lm_eval/tasks/afrobench/afrisenti/README.md -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:21,484 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:21,489 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/run.sh -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:21,493 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:21,499 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:21,503 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/run.sh -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:21,508 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:21,513 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:21,519 copying lm_eval/tasks/afrobench/xlsum/README.md -> build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T14:29:21,523 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:21,527 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:21,532 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:21,536 copying lm_eval/tasks/afrobench/openai_mmlu/README.md -> build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:29:21,541 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:21,544 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:21,548 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:21,551 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:21,554 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:21,558 copying lm_eval/tasks/afrobench/uhura-arc-easy/README.md -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:29:21,562 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:21,565 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:21,568 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:21,572 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:21,575 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:21,578 copying lm_eval/tasks/afrobench/masakhapos/README.md -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:29:21,583 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:21,587 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:21,590 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:21,593 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:21,597 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:21,599 copying lm_eval/tasks/afrobench/flores/README.md -> build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T14:29:21,603 copying lm_eval/tasks/afrobench/flores/prompt_1/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1 2026-05-11T14:29:21,606 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:21,608 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:21,611 copying lm_eval/tasks/afrobench/flores/prompt_2/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2 2026-05-11T14:29:21,614 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:21,617 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:21,620 copying lm_eval/tasks/afrobench/flores/prompt_3/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3 2026-05-11T14:29:21,622 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:21,625 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:21,628 copying lm_eval/tasks/afrobench/adr/README.md -> build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T14:29:21,632 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:21,635 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:21,638 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:21,642 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:21,644 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:21,648 copying lm_eval/tasks/afrobench/belebele/README.md -> build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T14:29:21,651 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:21,654 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:21,657 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:21,660 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:21,662 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:21,666 copying lm_eval/tasks/afrobench/mafand/README.md -> build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T14:29:21,669 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:21,673 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:21,677 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:21,680 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:21,684 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:21,687 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:21,690 copying lm_eval/tasks/afrobench/sib/README.md -> build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T14:29:21,694 copying lm_eval/tasks/afrobench/sib/prompt_1/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:21,697 copying lm_eval/tasks/afrobench/sib/prompt_5/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:21,701 copying lm_eval/tasks/afrobench/sib/prompt_2/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:21,704 copying lm_eval/tasks/afrobench/sib/prompt_3/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:21,707 copying lm_eval/tasks/afrobench/sib/prompt_4/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:21,711 copying lm_eval/tasks/afrobench/masakhaner/README.md -> build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:29:21,714 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:21,717 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:21,721 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:21,724 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:21,728 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:21,731 copying lm_eval/tasks/afrobench/nollysenti/README.md -> build/lib/lm_eval/tasks/afrobench/nollysenti 2026-05-11T14:29:21,734 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:21,738 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:21,741 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:21,745 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:21,748 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:21,751 copying lm_eval/tasks/afrobench/afriqa/README.md -> build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T14:29:21,755 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:21,759 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:21,762 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:21,765 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:21,768 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:21,771 copying lm_eval/tasks/afrobench/injongointent/README.md -> build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T14:29:21,775 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:21,779 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:21,782 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:21,785 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:21,788 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:21,791 copying lm_eval/tasks/afrobench/salt/README.md -> build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T14:29:21,794 copying lm_eval/tasks/afrobench/salt/prompt_1/salt -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:21,797 copying lm_eval/tasks/afrobench/salt/prompt_2/salt -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:21,800 copying lm_eval/tasks/afrobench/salt/prompt_3/salt -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:21,802 copying lm_eval/tasks/afrobench/masakhanews/README.md -> build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:29:21,807 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:21,810 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:21,813 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:21,816 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:21,820 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:21,822 copying lm_eval/tasks/xquad/xquad_common_yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:21,825 copying lm_eval/tasks/xquad/README.md -> build/lib/lm_eval/tasks/xquad 2026-05-11T14:29:21,828 copying lm_eval/tasks/spanish_bench/README.md -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T14:29:21,832 copying lm_eval/tasks/spanish_bench/phrases_es/_phrases_es_common -> build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:21,835 copying lm_eval/tasks/spanish_bench/flores_es/_flores_common_yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:21,838 copying lm_eval/tasks/paloma/_paloma_template -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:21,841 copying lm_eval/tasks/paloma/README.md -> build/lib/lm_eval/tasks/paloma 2026-05-11T14:29:21,844 copying lm_eval/tasks/translation/wmt_common_yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:21,847 copying lm_eval/tasks/translation/README.md -> build/lib/lm_eval/tasks/translation 2026-05-11T14:29:21,850 copying lm_eval/tasks/truthfulqa-multi/README.md -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:21,853 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc_common -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:21,856 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_common -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:21,859 copying lm_eval/tasks/cabbq/README.md -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:21,862 copying lm_eval/tasks/cabbq/_cabbq_common_yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T14:29:21,866 copying lm_eval/tasks/groundcocoa/README.md -> build/lib/lm_eval/tasks/groundcocoa 2026-05-11T14:29:21,870 copying lm_eval/tasks/fda/README.md -> build/lib/lm_eval/tasks/fda 2026-05-11T14:29:21,873 copying lm_eval/tasks/squadv2/README.md -> build/lib/lm_eval/tasks/squadv2 2026-05-11T14:29:21,876 copying lm_eval/tasks/aime/README.md -> build/lib/lm_eval/tasks/aime 2026-05-11T14:29:21,879 copying lm_eval/tasks/french_bench/_default_template_yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:21,883 copying lm_eval/tasks/french_bench/README.md -> build/lib/lm_eval/tasks/french_bench 2026-05-11T14:29:21,886 copying lm_eval/tasks/eq_bench/README.md -> build/lib/lm_eval/tasks/eq_bench 2026-05-11T14:29:21,890 copying lm_eval/tasks/esbbq/README.md -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:21,893 copying lm_eval/tasks/esbbq/_esbbq_common_yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T14:29:21,896 copying lm_eval/tasks/arc/README.md -> build/lib/lm_eval/tasks/arc 2026-05-11T14:29:21,899 copying lm_eval/tasks/bhs/README.md -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:21,902 copying lm_eval/tasks/bhs/_template_yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T14:29:21,905 copying lm_eval/tasks/mmlu-redux/generative/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:21,907 copying lm_eval/tasks/mmlu-redux/generative/README.md -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:21,910 copying lm_eval/tasks/lambada_cloze/README.md -> build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T14:29:21,912 copying lm_eval/tasks/webqs/README.md -> build/lib/lm_eval/tasks/webqs 2026-05-11T14:29:21,915 copying lm_eval/tasks/benchmarks/README.md -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T14:29:21,918 copying lm_eval/tasks/benchmarks/flan/_held_in_template_yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:21,921 copying lm_eval/tasks/benchmarks/multimedqa/README.md -> build/lib/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T14:29:21,923 copying lm_eval/tasks/lambada/README.md -> build/lib/lm_eval/tasks/lambada 2026-05-11T14:29:21,926 copying lm_eval/tasks/gsm8k/README.md -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T14:29:21,928 copying lm_eval/tasks/mts_dialog/README.md -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T14:29:21,932 copying lm_eval/tasks/darijammlu/README.md -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:21,936 copying lm_eval/tasks/darijammlu/_default_darijammlu_template_yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T14:29:21,938 copying lm_eval/tasks/truthfulqa/README.md -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T14:29:21,942 copying lm_eval/tasks/meddialog/README.md -> build/lib/lm_eval/tasks/meddialog 2026-05-11T14:29:21,945 copying lm_eval/tasks/histoires_morales/README.md -> build/lib/lm_eval/tasks/histoires_morales 2026-05-11T14:29:21,949 copying lm_eval/tasks/gsm_plus/README.md -> build/lib/lm_eval/tasks/gsm_plus 2026-05-11T14:29:21,951 copying lm_eval/tasks/darijahellaswag/README.md -> build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T14:29:21,954 copying lm_eval/tasks/kormedmcqa/README.md -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:21,957 copying lm_eval/tasks/kormedmcqa/_template_yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:21,959 copying lm_eval/tasks/arc_mt/README.md -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T14:29:21,962 copying lm_eval/tasks/mmlu-pro-plus/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:21,964 copying lm_eval/tasks/mmlu-pro-plus/README.md -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:21,968 copying lm_eval/tasks/wsc273/README.md -> build/lib/lm_eval/tasks/wsc273 2026-05-11T14:29:21,971 copying lm_eval/tasks/turblimp/README.md -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:21,973 copying lm_eval/tasks/turblimp/_template_yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T14:29:21,976 copying lm_eval/tasks/lambada_multilingual/README.md -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:21,978 copying lm_eval/tasks/score/README.md -> build/lib/lm_eval/tasks/score 2026-05-11T14:29:21,982 copying lm_eval/tasks/score/NON_GREEDY.md -> build/lib/lm_eval/tasks/score 2026-05-11T14:29:21,985 copying lm_eval/tasks/score/non_greedy.sh -> build/lib/lm_eval/tasks/score 2026-05-11T14:29:21,989 copying lm_eval/tasks/score/agi_eval/prompt_templates.json -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:21,992 copying lm_eval/tasks/score/math/to_be_fixed_questions.json -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:21,995 copying lm_eval/tasks/score/math/prompt_templates.json -> build/lib/lm_eval/tasks/score/math 2026-05-11T14:29:21,999 copying lm_eval/tasks/score/mmlu_pro/prompt_templates.json -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T14:29:22,002 copying lm_eval/tasks/mmlu/README.md -> build/lib/lm_eval/tasks/mmlu 2026-05-11T14:29:22,005 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:22,012 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:29:22,015 copying lm_eval/tasks/mmlu/generative/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T14:29:22,017 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:29:22,021 copying lm_eval/tasks/mmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T14:29:22,023 copying lm_eval/tasks/mmlu/continuation/_continuation_template_yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T14:29:22,029 copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:29:22,032 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:29:22,034 copying lm_eval/tasks/evalita_llm/_re_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,037 copying lm_eval/tasks/evalita_llm/_sa_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,039 copying lm_eval/tasks/evalita_llm/_hs_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,042 copying lm_eval/tasks/evalita_llm/_ls_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,045 copying lm_eval/tasks/evalita_llm/_sa_template_v2_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,048 copying lm_eval/tasks/evalita_llm/README.md -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,050 copying lm_eval/tasks/evalita_llm/_faq_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,053 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,055 copying lm_eval/tasks/evalita_llm/_te_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,058 copying lm_eval/tasks/evalita_llm/_sum_template_fp_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,060 copying lm_eval/tasks/evalita_llm/_sum_template_fp-small_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,063 copying lm_eval/tasks/evalita_llm/_at_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,066 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,068 copying lm_eval/tasks/evalita_llm/_sum_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,071 copying lm_eval/tasks/evalita_llm/_wic_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,073 copying lm_eval/tasks/evalita_llm/_ner_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,077 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T14:29:22,079 copying lm_eval/tasks/kbl/README.md -> build/lib/lm_eval/tasks/kbl 2026-05-11T14:29:22,083 copying lm_eval/tasks/kbl/knowledge/_kbl_knowledge_yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T14:29:22,085 copying lm_eval/tasks/kbl/bar_exam/criminal/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:29:22,088 copying lm_eval/tasks/kbl/bar_exam/public/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:29:22,091 copying lm_eval/tasks/kbl/bar_exam/responsibility/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:29:22,094 copying lm_eval/tasks/kbl/bar_exam/civil/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:29:22,096 copying lm_eval/tasks/kbl/reasoning/_kbl_reasoning_yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T14:29:22,099 copying lm_eval/tasks/lingoly/README.md -> build/lib/lm_eval/tasks/lingoly 2026-05-11T14:29:22,103 copying lm_eval/tasks/hendrycks_math/README.md -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T14:29:22,106 copying lm_eval/tasks/noreval/README.md -> build/lib/lm_eval/tasks/noreval 2026-05-11T14:29:22,109 copying lm_eval/tasks/noreval/noreval.jpg -> build/lib/lm_eval/tasks/noreval 2026-05-11T14:29:22,127 copying lm_eval/tasks/noreval/norec/_norec_yaml -> build/lib/lm_eval/tasks/noreval/norec 2026-05-11T14:29:22,131 copying lm_eval/tasks/noreval/norquad/_norquad_yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T14:29:22,134 copying lm_eval/tasks/noreval/noropenbookqa/_noropenbookqa_yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T14:29:22,137 copying lm_eval/tasks/noreval/tatoeba/_tatoeba_yaml -> build/lib/lm_eval/tasks/noreval/tatoeba 2026-05-11T14:29:22,140 copying lm_eval/tasks/noreval/norsumm/_norsumm_yaml -> build/lib/lm_eval/tasks/noreval/norsumm 2026-05-11T14:29:22,143 copying lm_eval/tasks/noreval/norcommonsenseqa/_norcommonsenseqa_yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa 2026-05-11T14:29:22,146 copying lm_eval/tasks/noreval/ask_gec/README.md -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:29:22,148 copying lm_eval/tasks/noreval/ask_gec/_ask_gec_yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:29:22,151 copying lm_eval/tasks/noreval/norbelebele/_norbelebele_yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:29:22,154 copying lm_eval/tasks/noreval/noridiom/_noridiom_yaml -> build/lib/lm_eval/tasks/noreval/noridiom 2026-05-11T14:29:22,157 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/_nortruthfulqa_mc_yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice 2026-05-11T14:29:22,162 copying lm_eval/tasks/noreval/nortruthfulqa/generation/_nortruthfulqa_gen_yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T14:29:22,164 copying lm_eval/tasks/noreval/nrk_quiz_qa/_nrk_quiz_qa_yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa 2026-05-11T14:29:22,169 copying lm_eval/tasks/mmlusr/README.md -> build/lib/lm_eval/tasks/mmlusr 2026-05-11T14:29:22,171 copying lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:29:22,174 copying lm_eval/tasks/mmlusr/answer_only/_mmlusr_a_yml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:29:22,178 copying lm_eval/tasks/mmlusr/question_and_answer/_mmlusr_qna_yml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:29:22,180 copying lm_eval/tasks/egyhellaswag/README.md -> build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T14:29:22,184 copying lm_eval/tasks/wikitext/README.md -> build/lib/lm_eval/tasks/wikitext 2026-05-11T14:29:22,186 copying lm_eval/tasks/mmlu_pro/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:22,188 copying lm_eval/tasks/mmlu_pro/README.md -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T14:29:22,192 copying lm_eval/tasks/eus_exams/eus_exams_es -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:22,194 copying lm_eval/tasks/eus_exams/README.md -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:22,197 copying lm_eval/tasks/eus_exams/eus_exams_eu -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:22,200 copying lm_eval/tasks/eus_exams/eus_exams -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T14:29:32,291 installing to build/bdist.linux-armv7l/wheel 2026-05-11T14:29:32,292 running install 2026-05-11T14:29:32,316 running install_lib 2026-05-11T14:29:32,322 creating build/bdist.linux-armv7l/wheel 2026-05-11T14:29:32,324 creating build/bdist.linux-armv7l/wheel/lm_eval 2026-05-11T14:29:32,326 creating build/bdist.linux-armv7l/wheel/lm_eval/caching 2026-05-11T14:29:32,327 copying build/lib/lm_eval/caching/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/caching 2026-05-11T14:29:32,328 copying build/lib/lm_eval/caching/cache.py -> build/bdist.linux-armv7l/wheel/./lm_eval/caching 2026-05-11T14:29:32,331 creating build/bdist.linux-armv7l/wheel/lm_eval/api 2026-05-11T14:29:32,332 copying build/lib/lm_eval/api/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,337 copying build/lib/lm_eval/api/model.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,339 copying build/lib/lm_eval/api/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,342 copying build/lib/lm_eval/api/instance.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,344 copying build/lib/lm_eval/api/registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,347 copying build/lib/lm_eval/api/samplers.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,349 copying build/lib/lm_eval/api/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,351 copying build/lib/lm_eval/api/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,352 copying build/lib/lm_eval/api/filter.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,354 copying build/lib/lm_eval/api/group.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T14:29:32,357 creating build/bdist.linux-armv7l/wheel/lm_eval/decontamination 2026-05-11T14:29:32,358 copying build/lib/lm_eval/decontamination/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T14:29:32,360 copying build/lib/lm_eval/decontamination/archiver.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T14:29:32,362 copying build/lib/lm_eval/decontamination/decontaminate.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T14:29:32,364 copying build/lib/lm_eval/decontamination/janitor.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T14:29:32,367 creating build/bdist.linux-armv7l/wheel/lm_eval/prompts 2026-05-11T14:29:32,368 copying build/lib/lm_eval/prompts/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/prompts 2026-05-11T14:29:32,374 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks 2026-05-11T14:29:32,376 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue 2026-05-11T14:29:32,377 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli 2026-05-11T14:29:32,378 copying build/lib/lm_eval/tasks/glue/mnli/mismatch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mnli 2026-05-11T14:29:32,380 copying build/lib/lm_eval/tasks/glue/mnli/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mnli 2026-05-11T14:29:32,382 copying build/lib/lm_eval/tasks/glue/mnli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mnli 2026-05-11T14:29:32,384 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mrpc 2026-05-11T14:29:32,385 copying build/lib/lm_eval/tasks/glue/mrpc/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mrpc 2026-05-11T14:29:32,387 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qnli 2026-05-11T14:29:32,388 copying build/lib/lm_eval/tasks/glue/qnli/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/qnli 2026-05-11T14:29:32,391 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/cola 2026-05-11T14:29:32,392 copying build/lib/lm_eval/tasks/glue/cola/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/cola 2026-05-11T14:29:32,394 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/rte 2026-05-11T14:29:32,395 copying build/lib/lm_eval/tasks/glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/rte 2026-05-11T14:29:32,398 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/sst2 2026-05-11T14:29:32,399 copying build/lib/lm_eval/tasks/glue/sst2/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/sst2 2026-05-11T14:29:32,400 copying build/lib/lm_eval/tasks/glue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue 2026-05-11T14:29:32,403 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qqp 2026-05-11T14:29:32,404 copying build/lib/lm_eval/tasks/glue/qqp/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/qqp 2026-05-11T14:29:32,406 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/wnli 2026-05-11T14:29:32,407 copying build/lib/lm_eval/tasks/glue/wnli/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/wnli 2026-05-11T14:29:32,409 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,410 copying build/lib/lm_eval/tasks/nerel-bench/ru_relation_definition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,413 copying build/lib/lm_eval/tasks/nerel-bench/ru_entity_definition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,415 copying build/lib/lm_eval/tasks/nerel-bench/ru_entity_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,417 copying build/lib/lm_eval/tasks/nerel-bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,420 copying build/lib/lm_eval/tasks/nerel-bench/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,423 copying build/lib/lm_eval/tasks/nerel-bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,426 copying build/lib/lm_eval/tasks/nerel-bench/ru_relation_extraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nerel-bench 2026-05-11T14:29:32,429 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/med_prescriptions 2026-05-11T14:29:32,430 copying build/lib/lm_eval/tasks/med_prescriptions/med_prescriptions_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_prescriptions 2026-05-11T14:29:32,432 copying build/lib/lm_eval/tasks/med_prescriptions/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_prescriptions 2026-05-11T14:29:32,436 copying build/lib/lm_eval/tasks/med_prescriptions/med_prescriptions_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_prescriptions 2026-05-11T14:29:32,438 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2026-05-11T14:29:32,439 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T14:29:32,441 copying build/lib/lm_eval/tasks/xnli_eu/xnli_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T14:29:32,443 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu_native.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T14:29:32,446 copying build/lib/lm_eval/tasks/xnli_eu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T14:29:32,449 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T14:29:32,453 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande 2026-05-11T14:29:32,454 copying build/lib/lm_eval/tasks/winogrande/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogrande 2026-05-11T14:29:32,455 copying build/lib/lm_eval/tasks/winogrande/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogrande 2026-05-11T14:29:32,457 copying build/lib/lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogrande 2026-05-11T14:29:32,460 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,461 copying build/lib/lm_eval/tasks/xstorycloze/default_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,463 copying build/lib/lm_eval/tasks/xstorycloze/default_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,464 copying build/lib/lm_eval/tasks/xstorycloze/default_my.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,466 copying build/lib/lm_eval/tasks/xstorycloze/default_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,468 copying build/lib/lm_eval/tasks/xstorycloze/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,470 copying build/lib/lm_eval/tasks/xstorycloze/default_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,471 copying build/lib/lm_eval/tasks/xstorycloze/default_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,473 copying build/lib/lm_eval/tasks/xstorycloze/default_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,475 copying build/lib/lm_eval/tasks/xstorycloze/default_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,477 copying build/lib/lm_eval/tasks/xstorycloze/_xstorycloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,479 copying build/lib/lm_eval/tasks/xstorycloze/default_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,481 copying build/lib/lm_eval/tasks/xstorycloze/default_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,482 copying build/lib/lm_eval/tasks/xstorycloze/default_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T14:29:32,485 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2026-05-11T14:29:32,486 copying build/lib/lm_eval/tasks/haerae/haerae_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,488 copying build/lib/lm_eval/tasks/haerae/haerae_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,490 copying build/lib/lm_eval/tasks/haerae/haerae_lw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,492 copying build/lib/lm_eval/tasks/haerae/_haerae.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,494 copying build/lib/lm_eval/tasks/haerae/haerae_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,496 copying build/lib/lm_eval/tasks/haerae/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,498 copying build/lib/lm_eval/tasks/haerae/_default_haerae_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,500 copying build/lib/lm_eval/tasks/haerae/haerae_gk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T14:29:32,502 copying build/lib/lm_eval/tasks/manager.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T14:29:32,505 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/humaneval 2026-05-11T14:29:32,506 copying build/lib/lm_eval/tasks/humaneval/humaneval_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T14:29:32,508 copying build/lib/lm_eval/tasks/humaneval/humaneval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T14:29:32,510 copying build/lib/lm_eval/tasks/humaneval/humaneval_64_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T14:29:32,511 copying build/lib/lm_eval/tasks/humaneval/humaneval_64.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T14:29:32,513 copying build/lib/lm_eval/tasks/humaneval/humaneval_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T14:29:32,515 copying build/lib/lm_eval/tasks/humaneval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T14:29:32,518 copying build/lib/lm_eval/tasks/humaneval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T14:29:32,521 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_proficiency 2026-05-11T14:29:32,522 copying build/lib/lm_eval/tasks/eus_proficiency/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_proficiency 2026-05-11T14:29:32,524 copying build/lib/lm_eval/tasks/eus_proficiency/eus_proficiency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_proficiency 2026-05-11T14:29:32,527 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag 2026-05-11T14:29:32,528 copying build/lib/lm_eval/tasks/swag/swag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swag 2026-05-11T14:29:32,530 copying build/lib/lm_eval/tasks/swag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swag 2026-05-11T14:29:32,534 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2026-05-11T14:29:32,535 copying build/lib/lm_eval/tasks/scrolls/scrolls_qmsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,537 copying build/lib/lm_eval/tasks/scrolls/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,540 copying build/lib/lm_eval/tasks/scrolls/scrolls_quality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,541 copying build/lib/lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,543 copying build/lib/lm_eval/tasks/scrolls/scrolls_qasper.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,545 copying build/lib/lm_eval/tasks/scrolls/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,547 copying build/lib/lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,548 copying build/lib/lm_eval/tasks/scrolls/scrolls_contractnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,550 copying build/lib/lm_eval/tasks/scrolls/scrolls_govreport.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T14:29:32,552 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:29:32,554 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:29:32,555 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:29:32,557 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:29:32,559 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T14:29:32,563 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/multiblimp 2026-05-11T14:29:32,564 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,566 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ces.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,567 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,569 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_deu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,571 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ron.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,572 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_chu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,574 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_spa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,576 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_mkd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,578 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_uzb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,579 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_eus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,581 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sah.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,582 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_urd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,584 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_cym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,586 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_guj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,587 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_glg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,589 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kxh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,591 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_quc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,592 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_mdf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,594 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_swe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,596 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,597 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_abk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,599 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_heb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,601 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_wbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,603 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,604 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bel.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,606 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kmr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,608 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ukr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,609 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_tam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,611 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,613 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_pol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,614 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_azz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,616 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,618 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hye.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,620 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_aln.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,621 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lav.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,623 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_got.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,625 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_slv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,626 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_arb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,628 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,630 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,631 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sqi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,633 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_vep.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,634 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bre.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,636 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_est.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,638 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_urb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,639 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,641 copying build/lib/lm_eval/tasks/multiblimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,643 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_xnr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,645 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,646 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_slk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,648 copying build/lib/lm_eval/tasks/multiblimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,650 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_myv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,652 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_nds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,653 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,655 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hyw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,657 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,659 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_olo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,660 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_apu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,662 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hun.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,664 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_gla.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,665 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_tur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,667 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fas.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,669 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_cat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,670 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_yrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,672 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_rus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,674 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ttc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,675 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ell.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,677 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ota.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,679 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_krl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,680 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_aqz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,682 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_xpg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,684 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_dan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,685 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,687 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,689 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,690 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_orv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,692 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ben.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,694 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,695 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_xcl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,697 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_san.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,699 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,701 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_grc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,702 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,704 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_uig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,706 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,707 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_frm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,709 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_nhi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,711 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,713 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,714 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,716 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ita.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,718 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_gle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,719 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_mar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,721 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,723 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hsb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,724 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_isl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,726 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,728 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_koi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,729 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_nld.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,731 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kpv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,733 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_tpn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,734 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lij.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,736 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T14:29:32,739 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2026-05-11T14:29:32,740 copying build/lib/lm_eval/tasks/glianorex/glianorex_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T14:29:32,742 copying build/lib/lm_eval/tasks/glianorex/glianorex.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T14:29:32,743 copying build/lib/lm_eval/tasks/glianorex/glianorex_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T14:29:32,745 copying build/lib/lm_eval/tasks/glianorex/preprocess_glianorex.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T14:29:32,747 copying build/lib/lm_eval/tasks/glianorex/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T14:29:32,749 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/discrim_eval 2026-05-11T14:29:32,750 copying build/lib/lm_eval/tasks/discrim_eval/discrim_eval_explicit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T14:29:32,752 copying build/lib/lm_eval/tasks/discrim_eval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T14:29:32,754 copying build/lib/lm_eval/tasks/discrim_eval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T14:29:32,756 copying build/lib/lm_eval/tasks/discrim_eval/discrim_eval_implicit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T14:29:32,758 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2026-05-11T14:29:32,759 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T14:29:32,761 copying build/lib/lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T14:29:32,763 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T14:29:32,764 copying build/lib/lm_eval/tasks/qa4mre/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T14:29:32,766 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T14:29:32,768 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/race 2026-05-11T14:29:32,769 copying build/lib/lm_eval/tasks/race/race.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/race 2026-05-11T14:29:32,771 copying build/lib/lm_eval/tasks/race/preprocess_race.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/race 2026-05-11T14:29:32,772 copying build/lib/lm_eval/tasks/race/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/race 2026-05-11T14:29:32,775 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bear 2026-05-11T14:29:32,776 copying build/lib/lm_eval/tasks/bear/bear_big.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bear 2026-05-11T14:29:32,778 copying build/lib/lm_eval/tasks/bear/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bear 2026-05-11T14:29:32,779 copying build/lib/lm_eval/tasks/bear/bear.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bear 2026-05-11T14:29:32,782 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogender 2026-05-11T14:29:32,783 copying build/lib/lm_eval/tasks/winogender/winogender_male.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,785 copying build/lib/lm_eval/tasks/winogender/winogender_gotcha_male.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,786 copying build/lib/lm_eval/tasks/winogender/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,788 copying build/lib/lm_eval/tasks/winogender/winogender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,790 copying build/lib/lm_eval/tasks/winogender/winogender_neutral.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,792 copying build/lib/lm_eval/tasks/winogender/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,793 copying build/lib/lm_eval/tasks/winogender/winogender_female.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,795 copying build/lib/lm_eval/tasks/winogender/winogender_gotcha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,797 copying build/lib/lm_eval/tasks/winogender/winogender_gotcha_female.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T14:29:32,800 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2026-05-11T14:29:32,801 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,803 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,804 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,806 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,808 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,810 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,811 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,813 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,815 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,816 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,818 copying build/lib/lm_eval/tasks/bertaqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,820 copying build/lib/lm_eval/tasks/bertaqa/_bertaqa_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,821 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,823 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,824 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,826 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,828 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,829 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T14:29:32,832 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arab_culture 2026-05-11T14:29:32,833 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_levant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,835 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_uae.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,836 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_libya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,838 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_tunisia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,840 copying build/lib/lm_eval/tasks/arab_culture/prompts.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,842 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,843 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_algeria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,845 copying build/lib/lm_eval/tasks/arab_culture/utils_mcq.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,847 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,849 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,851 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,852 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,854 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_morocco.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,856 copying build/lib/lm_eval/tasks/arab_culture/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,858 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_sudan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,859 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_north_africa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,861 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_yemen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,863 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_nile_valley.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,864 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_ksa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,866 copying build/lib/lm_eval/tasks/arab_culture/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,868 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_gulf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,869 copying build/lib/lm_eval/tasks/arab_culture/_default_arab_culture_mcq_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,871 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T14:29:32,873 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_trivia 2026-05-11T14:29:32,874 copying build/lib/lm_eval/tasks/eus_trivia/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_trivia 2026-05-11T14:29:32,876 copying build/lib/lm_eval/tasks/eus_trivia/eus_trivia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_trivia 2026-05-11T14:29:32,878 copying build/lib/lm_eval/tasks/eus_trivia/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_trivia 2026-05-11T14:29:32,880 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T14:29:32,881 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:29:32,882 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_mt_openbook_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:29:32,884 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_leaderboard_arabic_mt_openbook_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:29:32,886 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T14:29:32,888 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:29:32,889 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_leaderboard_arabic_mt_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:29:32,891 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:29:32,893 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_mt_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T14:29:32,895 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:29:32,896 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_leaderboard_arabic_mt_boolq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:29:32,898 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_mt_boolq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:29:32,900 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T14:29:32,902 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:29:32,903 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_leaderboard_arabic_mt_copa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:29:32,905 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:29:32,907 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_mt_copa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T14:29:32,909 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:29:32,910 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_mt_arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:29:32,912 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_leaderboard_arabic_mt_arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:29:32,914 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T14:29:32,916 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:29:32,917 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_leaderboard_arabic_exams.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:29:32,919 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_exams.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:29:32,921 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T14:29:32,923 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_complete.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T14:29:32,925 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:29:32,926 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_mt_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:29:32,928 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:29:32,930 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_leaderboard_arabic_mt_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T14:29:32,932 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:29:32,933 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_mt_arc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:29:32,935 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_leaderboard_arabic_mt_arc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:29:32,936 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T14:29:32,940 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,941 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,942 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,944 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,946 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,948 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,950 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,951 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,953 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,955 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,957 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,958 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,960 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,962 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,964 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,965 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,967 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,969 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,970 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,972 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,974 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,975 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,977 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,979 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,981 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,982 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,984 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,986 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,988 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,989 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,991 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,993 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,995 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,997 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:32,998 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,000 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,002 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,003 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,005 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,007 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,009 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,011 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,012 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,014 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,016 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,017 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,019 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,021 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,022 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,024 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,026 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,028 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,030 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,032 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,034 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,035 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,037 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,039 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,041 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,042 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T14:29:33,044 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T14:29:33,048 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,049 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,051 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,053 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_mcq_exams_test_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,055 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_sentiment_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,057 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,059 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,061 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_dialects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,063 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,065 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,067 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,069 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T14:29:33,071 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:29:33,072 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_leaderboard_arabic_mt_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:29:33,074 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_mt_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:29:33,076 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T14:29:33,079 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:29:33,080 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_leaderboard_arabic_mt_race.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:29:33,082 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:29:33,084 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_mt_race.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T14:29:33,087 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,088 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,090 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,092 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mauritania.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,093 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Yemen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,095 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Comoros.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,097 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromRome.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,099 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Calligraphy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,100 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Morocco.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,102 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromGreece.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,104 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,106 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,107 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ornament.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,109 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_computer_and_phone.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,111 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Ancient_Egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,113 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,114 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Sudan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,116 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,118 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Iraq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,120 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromPersia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,121 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Language_Origin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,123 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,125 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,127 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ceremony.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,129 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Funeral.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,131 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Clothing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,132 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_daily_life.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,134 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Tunisia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,135 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_branches_and_schools.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,137 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Somalia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,139 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Algeria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,141 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_History.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,142 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,144 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Egypt_modern.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,146 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_Education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,148 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,149 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromIslam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,151 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Bahrain.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,153 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Wedding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,155 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mesopotamia_civilization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,157 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromAncientEgypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,158 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromChina.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,160 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,162 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Libya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,163 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arab_Empire.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,165 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Oman.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,167 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Qatar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,169 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_entertainment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,170 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,172 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,174 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromByzantium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,176 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_United_Arab_Emirates.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,177 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Saudi_Arabia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,179 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Art.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,181 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Food.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,183 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,184 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Kuwait.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,186 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_communication.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,188 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islamic_law_system.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,190 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,191 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T14:29:33,194 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:29:33,195 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_leaderboard_arabic_mt_sciq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:29:33,197 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_mt_sciq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:29:33,198 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T14:29:33,201 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:29:33,202 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_leaderboard_arabic_mt_toxigen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:29:33,204 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:29:33,206 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_mt_toxigen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T14:29:33,208 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench 2026-05-11T14:29:33,210 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,211 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/next_act.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,214 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,218 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/acp_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,221 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/_gen_yaml_2shot -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,223 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,225 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,227 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,229 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/acp_grammar.lark -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,230 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,232 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,234 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T14:29:33,237 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,238 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/_mcq_cot_2shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,239 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,242 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,244 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,246 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,248 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,250 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,252 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T14:29:33,254 copying build/lib/lm_eval/tasks/acpbench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench 2026-05-11T14:29:33,257 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,258 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,260 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/_boolq_cot_2shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,262 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,264 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,266 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,267 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,269 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,271 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T14:29:33,274 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,275 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/next_act.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,277 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,279 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,282 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/_gen_yaml_2shot -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,284 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,286 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,288 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,291 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_grammar.lark -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,292 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,295 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,297 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T14:29:33,300 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/jsonschema_bench 2026-05-11T14:29:33,301 copying build/lib/lm_eval/tasks/jsonschema_bench/jsonschema_bench_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T14:29:33,303 copying build/lib/lm_eval/tasks/jsonschema_bench/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T14:29:33,306 copying build/lib/lm_eval/tasks/jsonschema_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T14:29:33,308 copying build/lib/lm_eval/tasks/jsonschema_bench/jsonschema_bench_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T14:29:33,310 copying build/lib/lm_eval/tasks/jsonschema_bench/jsonschema_bench_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T14:29:33,312 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_reading 2026-05-11T14:29:33,313 copying build/lib/lm_eval/tasks/eus_reading/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_reading 2026-05-11T14:29:33,315 copying build/lib/lm_eval/tasks/eus_reading/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_reading 2026-05-11T14:29:33,317 copying build/lib/lm_eval/tasks/eus_reading/eus_reading.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_reading 2026-05-11T14:29:33,320 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2026-05-11T14:29:33,321 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,323 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,325 copying build/lib/lm_eval/tasks/aclue/_aclue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,327 copying build/lib/lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,328 copying build/lib/lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,330 copying build/lib/lm_eval/tasks/aclue/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,332 copying build/lib/lm_eval/tasks/aclue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,333 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,335 copying build/lib/lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,337 copying build/lib/lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,338 copying build/lib/lm_eval/tasks/aclue/aclue_couplet_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,340 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,342 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,344 copying build/lib/lm_eval/tasks/aclue/aclue_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,345 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,347 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,349 copying build/lib/lm_eval/tasks/aclue/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,351 copying build/lib/lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,353 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T14:29:33,355 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/careqa 2026-05-11T14:29:33,356 copying build/lib/lm_eval/tasks/careqa/careqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,358 copying build/lib/lm_eval/tasks/careqa/careqa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,360 copying build/lib/lm_eval/tasks/careqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,361 copying build/lib/lm_eval/tasks/careqa/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,363 copying build/lib/lm_eval/tasks/careqa/utils_open.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,365 copying build/lib/lm_eval/tasks/careqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,367 copying build/lib/lm_eval/tasks/careqa/careqa_open.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,369 copying build/lib/lm_eval/tasks/careqa/careqa_open_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T14:29:33,371 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/icelandic_winogrande 2026-05-11T14:29:33,372 copying build/lib/lm_eval/tasks/icelandic_winogrande/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/icelandic_winogrande 2026-05-11T14:29:33,374 copying build/lib/lm_eval/tasks/icelandic_winogrande/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/icelandic_winogrande 2026-05-11T14:29:33,377 copying build/lib/lm_eval/tasks/icelandic_winogrande/preprocess_winogrande.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/icelandic_winogrande 2026-05-11T14:29:33,380 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hrm8k 2026-05-11T14:29:33,381 copying build/lib/lm_eval/tasks/hrm8k/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k 2026-05-11T14:29:33,384 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,385 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_gsm8k_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,387 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_mmmlu_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,389 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,391 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_math_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,392 copying build/lib/lm_eval/tasks/hrm8k/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,394 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_omni_math_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,396 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_ksm_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,398 copying build/lib/lm_eval/tasks/hrm8k/en/_hrm8k_en_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T14:29:33,400 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,401 copying build/lib/lm_eval/tasks/hrm8k/default/_hrm8k_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,403 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,405 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_omni_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,406 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,408 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_mmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,410 copying build/lib/lm_eval/tasks/hrm8k/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,413 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,415 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_ksm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T14:29:33,417 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/copal_id 2026-05-11T14:29:33,418 copying build/lib/lm_eval/tasks/copal_id/colloquial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T14:29:33,420 copying build/lib/lm_eval/tasks/copal_id/standard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T14:29:33,422 copying build/lib/lm_eval/tasks/copal_id/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T14:29:33,424 copying build/lib/lm_eval/tasks/copal_id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T14:29:33,427 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,429 copying build/lib/lm_eval/tasks/arab_culture_completion/utils_completion.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,431 copying build/lib/lm_eval/tasks/arab_culture_completion/prompts.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,433 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_libya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,435 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_algeria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,436 copying build/lib/lm_eval/tasks/arab_culture_completion/_default_arab_culture_completion_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,438 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,440 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,441 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,443 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_gulf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,445 copying build/lib/lm_eval/tasks/arab_culture_completion/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,447 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,448 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_uae.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,450 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_yemen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,452 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_tunisia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,454 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_morocco.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,455 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_sudan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,457 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,459 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_ksa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,461 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_north_africa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,462 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,464 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_nile_valley.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,466 copying build/lib/lm_eval/tasks/arab_culture_completion/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,468 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_levant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T14:29:33,470 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2026-05-11T14:29:33,471 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,473 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,475 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,476 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,478 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,480 copying build/lib/lm_eval/tasks/arithmetic/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,482 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,483 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,485 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,487 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,488 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T14:29:33,491 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medtext 2026-05-11T14:29:33,492 copying build/lib/lm_eval/tasks/medtext/medtext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T14:29:33,494 copying build/lib/lm_eval/tasks/medtext/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T14:29:33,496 copying build/lib/lm_eval/tasks/medtext/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T14:29:33,497 copying build/lib/lm_eval/tasks/medtext/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T14:29:33,499 copying build/lib/lm_eval/tasks/medtext/medtext_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T14:29:33,502 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pisa 2026-05-11T14:29:33,503 copying build/lib/lm_eval/tasks/pisa/pisa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,505 copying build/lib/lm_eval/tasks/pisa/pisa_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,507 copying build/lib/lm_eval/tasks/pisa/pisa_ch_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,509 copying build/lib/lm_eval/tasks/pisa/pisa_ch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,510 copying build/lib/lm_eval/tasks/pisa/pisa_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,512 copying build/lib/lm_eval/tasks/pisa/pisa_es_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,514 copying build/lib/lm_eval/tasks/pisa/pisa_fr_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,515 copying build/lib/lm_eval/tasks/pisa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,517 copying build/lib/lm_eval/tasks/pisa/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,519 copying build/lib/lm_eval/tasks/pisa/pisa_it_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,521 copying build/lib/lm_eval/tasks/pisa/pisa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,523 copying build/lib/lm_eval/tasks/pisa/pisa_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,525 copying build/lib/lm_eval/tasks/pisa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,528 copying build/lib/lm_eval/tasks/pisa/pisa_de_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,529 copying build/lib/lm_eval/tasks/pisa/_pisa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,531 copying build/lib/lm_eval/tasks/pisa/_pisa_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,532 copying build/lib/lm_eval/tasks/pisa/pisa_en_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T14:29:33,535 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2026-05-11T14:29:33,536 copying build/lib/lm_eval/tasks/xnli/xnli_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,538 copying build/lib/lm_eval/tasks/xnli/xnli_bg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,539 copying build/lib/lm_eval/tasks/xnli/xnli_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,541 copying build/lib/lm_eval/tasks/xnli/xnli_ur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,543 copying build/lib/lm_eval/tasks/xnli/xnli_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,544 copying build/lib/lm_eval/tasks/xnli/xnli_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,546 copying build/lib/lm_eval/tasks/xnli/xnli_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,548 copying build/lib/lm_eval/tasks/xnli/xnli_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,550 copying build/lib/lm_eval/tasks/xnli/_xnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,551 copying build/lib/lm_eval/tasks/xnli/xnli_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,553 copying build/lib/lm_eval/tasks/xnli/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,555 copying build/lib/lm_eval/tasks/xnli/xnli_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,557 copying build/lib/lm_eval/tasks/xnli/xnli_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,559 copying build/lib/lm_eval/tasks/xnli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,561 copying build/lib/lm_eval/tasks/xnli/xnli_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,563 copying build/lib/lm_eval/tasks/xnli/xnli_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,565 copying build/lib/lm_eval/tasks/xnli/xnli_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,567 copying build/lib/lm_eval/tasks/xnli/xnli_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,568 copying build/lib/lm_eval/tasks/xnli/xnli_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T14:29:33,571 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:33,572 copying build/lib/lm_eval/tasks/openai-mmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:33,574 copying build/lib/lm_eval/tasks/openai-mmmlu/subjects.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:33,588 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,589 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,591 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,593 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,595 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,596 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,598 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,600 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,601 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,603 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,605 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,607 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,609 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,611 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,612 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,614 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,616 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,618 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,620 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,622 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,624 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,626 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,628 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,630 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,632 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,634 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,636 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,637 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,639 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,641 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,642 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,644 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,646 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,648 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,649 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,651 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,653 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,655 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,657 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,659 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,660 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,662 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,664 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,666 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,668 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,670 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,671 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,673 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,675 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,677 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,678 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,680 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,682 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,683 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,685 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,686 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,688 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,690 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,692 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,694 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,695 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,697 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,699 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,701 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,703 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,704 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,706 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,708 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,710 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,712 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,714 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,715 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,717 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,719 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,720 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,722 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,724 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,726 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,727 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,729 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,731 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,733 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,735 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,736 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,738 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,740 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,742 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,744 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,746 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,748 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,750 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,752 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,754 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,755 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,757 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,759 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,760 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,762 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,763 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,765 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,767 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,768 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,770 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,772 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,774 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,776 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,777 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,779 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,781 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,783 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,785 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,786 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,788 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,790 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,792 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,794 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,796 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,797 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,799 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,801 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,803 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,804 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,806 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,808 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,810 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,811 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,813 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,815 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,817 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,819 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,821 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,822 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,824 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,826 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,828 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,829 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,831 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,833 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,835 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,836 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,838 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,840 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,842 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,843 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,845 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,847 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,849 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,851 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,852 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,854 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,856 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,858 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,859 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,861 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,863 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,865 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,867 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,869 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,871 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,873 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,874 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,876 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,878 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,879 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,881 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,883 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,884 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,886 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,888 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,889 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,891 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,893 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,895 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,897 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,898 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,900 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,902 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,904 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,905 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,907 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,909 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,911 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,912 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,914 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,916 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,917 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,919 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,921 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,922 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,924 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,926 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,928 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,929 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,931 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,933 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,934 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,936 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,938 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,940 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,942 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,944 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,945 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,948 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,950 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,952 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,954 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,955 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,957 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,959 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,961 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,963 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,964 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,966 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,968 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,970 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,972 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,975 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,977 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,978 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,980 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,982 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,984 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,986 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,987 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,989 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,991 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,992 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,994 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,996 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,997 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:33,999 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,001 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,002 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,004 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,006 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,008 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,009 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,011 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,013 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,015 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,017 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,018 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,020 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,023 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,025 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,027 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,029 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,030 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,032 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,034 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,035 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,037 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,038 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,040 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,042 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,043 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,045 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,047 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,049 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,050 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,052 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,054 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,055 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,057 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,059 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,060 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,062 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,064 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,065 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,067 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,069 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,070 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,072 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,074 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,075 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,077 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,079 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,080 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,082 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,084 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,086 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,087 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,089 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,091 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,093 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,095 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,097 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,098 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,100 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,102 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,104 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,106 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,107 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,109 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,111 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,112 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,114 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,116 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,117 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,119 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,121 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,123 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,124 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,126 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,128 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,130 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,132 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,134 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,135 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,137 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,139 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,141 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,143 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,144 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,146 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,148 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,150 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,152 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,153 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,155 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,157 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,158 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,160 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,162 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,163 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,165 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,167 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,168 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,170 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,172 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,174 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,176 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,177 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,179 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,181 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,183 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,185 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,188 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,189 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,191 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,193 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,195 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,197 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,199 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,201 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,203 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,205 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,207 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,208 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,210 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,212 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,214 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,215 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,217 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,219 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,221 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,224 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,225 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,227 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,229 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,231 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,232 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,234 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,236 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,237 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,239 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,241 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,243 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,245 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,246 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,248 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,250 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,252 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,254 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,256 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,258 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,260 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,262 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,264 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,266 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,269 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,271 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,272 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,274 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,276 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,277 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,279 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,281 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,282 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,284 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,286 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,288 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,290 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,291 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,293 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,295 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,297 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,299 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,302 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,304 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,307 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,310 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,312 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,313 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,315 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,317 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,318 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,320 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,322 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,324 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,325 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,327 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,329 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,331 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,333 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,334 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,336 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,338 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,340 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,341 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,343 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,345 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,347 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,349 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,351 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,352 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,354 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,356 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,357 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,359 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,361 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,362 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,364 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,366 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,367 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,369 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,371 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,373 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,374 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,377 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,379 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,381 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,383 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,385 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,387 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,389 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,391 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,393 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,395 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,396 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,398 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,400 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,401 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,403 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,405 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,406 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,408 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,410 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,412 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,413 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,415 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,417 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,419 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,421 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,423 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,424 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,427 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,428 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,430 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,432 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,434 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,435 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,437 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,439 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,440 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,442 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,444 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,446 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,448 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,449 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,452 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,454 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,456 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,457 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,459 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,461 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,463 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,465 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,467 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,469 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,471 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,473 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,475 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,476 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,478 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,480 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,482 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,484 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,486 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,488 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,490 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,492 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,494 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,496 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,498 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,500 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,502 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,504 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,506 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,508 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,509 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,511 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,513 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,514 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,516 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,518 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,519 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,521 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,523 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,524 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,526 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,528 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,529 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,531 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,533 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,534 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,536 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,538 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,539 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,541 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,542 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,544 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,546 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,548 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,549 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,551 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,553 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,555 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,556 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,558 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,560 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,561 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,563 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,564 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,566 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,568 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,570 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,571 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,573 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,575 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,577 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,579 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,580 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,582 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,584 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,586 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,587 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,589 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,591 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,593 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,594 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,596 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,597 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,599 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,601 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,602 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,604 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,605 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,607 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,609 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,611 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,613 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,614 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,616 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,618 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,620 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,621 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,623 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,625 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,627 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,628 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,630 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,632 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,633 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,635 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,637 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,638 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,640 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,642 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,643 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,645 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,647 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,649 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,651 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,652 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,654 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,656 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,658 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,660 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,662 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,664 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,666 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,668 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,670 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,671 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,673 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,675 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,677 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,678 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,681 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,682 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,684 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,686 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,688 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,690 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,691 copying build/lib/lm_eval/tasks/openai-mmmlu/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,693 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,695 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,697 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,699 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,701 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,703 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,705 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,707 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,709 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,711 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,712 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,714 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,716 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,718 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,720 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,722 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,724 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,726 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,728 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,730 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,732 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,733 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,735 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,737 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,739 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,741 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,743 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,745 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,748 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,750 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,752 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,754 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,755 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,758 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,760 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,762 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,764 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,766 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,768 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,770 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,772 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,774 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,776 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,778 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,780 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,782 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,784 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,787 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,789 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,791 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,793 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,796 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,797 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,800 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,802 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,804 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,806 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,807 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,809 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,811 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,813 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,815 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,817 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,819 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,821 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,822 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,824 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,826 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,828 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,830 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,832 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,834 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,836 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,838 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,840 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,842 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,845 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,847 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,848 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,850 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,852 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,855 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,857 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,858 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,860 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,862 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,864 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,866 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,868 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,870 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,872 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,875 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,877 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,879 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,881 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,883 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,885 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,887 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,890 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,892 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,894 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,896 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,898 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,901 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,903 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,905 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,908 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,909 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,911 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,913 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,915 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,917 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,919 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,921 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,923 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,925 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,927 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,929 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,931 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,934 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,936 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,938 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,940 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,943 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,945 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,947 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,949 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,951 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,953 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,955 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,957 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,959 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,961 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,962 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,965 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,967 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,969 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,971 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,973 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,974 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,977 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,979 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,981 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,983 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,984 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,986 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,988 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,990 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,991 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,993 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,995 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,997 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:34,999 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,001 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,002 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,004 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,006 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,008 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,010 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,012 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,014 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,015 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,017 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,019 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,021 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,023 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,025 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,027 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,029 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,030 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,032 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,034 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,036 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,037 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,039 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,041 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,043 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,045 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,046 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,048 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,050 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,052 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,054 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,055 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,057 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,059 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,061 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,063 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,065 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,067 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,068 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,070 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,072 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,073 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,075 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,077 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,079 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,081 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,083 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,084 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,086 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,088 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,090 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,091 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,093 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,095 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,097 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,099 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,101 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,104 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,106 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,108 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,110 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,111 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,113 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,115 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,117 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,119 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,121 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,123 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,125 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,127 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,128 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,130 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,132 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,134 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,136 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,138 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,140 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,142 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,144 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,146 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,148 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,150 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,152 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,154 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,156 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,157 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,159 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,161 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,163 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,165 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,167 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,169 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,171 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,173 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,175 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,177 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,179 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,181 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,183 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,185 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,187 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,189 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,191 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,193 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T14:29:35,195 copying build/lib/lm_eval/tasks/openai-mmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:35,197 copying build/lib/lm_eval/tasks/openai-mmmlu/languages.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T14:29:35,199 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/meqsum 2026-05-11T14:29:35,201 copying build/lib/lm_eval/tasks/meqsum/meqsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meqsum 2026-05-11T14:29:35,203 copying build/lib/lm_eval/tasks/meqsum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meqsum 2026-05-11T14:29:35,204 copying build/lib/lm_eval/tasks/meqsum/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meqsum 2026-05-11T14:29:35,207 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze 2026-05-11T14:29:35,208 copying build/lib/lm_eval/tasks/storycloze/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/storycloze 2026-05-11T14:29:35,210 copying build/lib/lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/storycloze 2026-05-11T14:29:35,212 copying build/lib/lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/storycloze 2026-05-11T14:29:35,215 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu 2026-05-11T14:29:35,217 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full 2026-05-11T14:29:35,219 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,221 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,223 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,224 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,226 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,228 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,230 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,232 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,234 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,235 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,238 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,240 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,242 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,244 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,246 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,248 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,249 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,251 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,253 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,254 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,256 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_de_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,258 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,259 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,261 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,263 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,265 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,267 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,268 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,270 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,272 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,273 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,275 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,277 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,279 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,281 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,283 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,284 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,286 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,288 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,290 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,291 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,293 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,295 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,296 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,298 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,300 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,301 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,303 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,305 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,307 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,309 copying build/lib/lm_eval/tasks/global_mmlu/full/de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,311 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,313 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,315 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,317 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,319 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,320 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,322 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,324 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,326 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,327 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,329 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,331 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,332 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,334 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T14:29:35,337 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,338 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,340 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,342 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,343 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,345 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,347 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,349 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,350 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,352 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,354 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,356 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,357 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,359 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,368 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,369 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,371 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,372 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,374 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,376 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,377 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,379 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,381 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,382 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,384 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,386 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,387 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,389 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,391 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,393 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,394 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,396 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,398 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,400 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,401 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,403 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,405 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,406 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,408 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,409 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,411 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,413 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,414 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,416 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,418 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,419 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,421 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,423 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,424 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,426 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,428 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,430 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,431 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,433 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,435 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,437 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,439 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,440 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,442 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,444 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_hi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,445 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,447 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,449 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,450 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,452 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T14:29:35,455 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,457 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,459 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,460 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,462 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,464 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,466 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,467 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,469 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,471 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,474 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,476 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,478 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,480 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,482 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,484 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,486 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,488 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,490 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,492 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,494 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,496 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,497 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,499 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,501 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,502 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,504 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,506 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,508 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,510 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,512 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,513 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,515 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,517 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,519 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,521 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,523 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,524 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,526 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,528 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,530 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_bn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,531 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,533 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,535 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,536 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,538 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,540 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,542 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,544 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,546 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,548 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,550 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,552 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,553 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,555 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,557 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,559 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,561 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,563 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,565 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,566 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,568 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,570 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,571 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,573 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T14:29:35,576 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,577 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,579 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,581 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,583 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,585 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,587 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,589 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,591 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,592 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,594 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,596 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,598 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,600 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,602 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,603 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,605 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,606 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,608 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,610 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,612 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,613 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,616 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,617 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,619 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,621 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,623 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,624 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,626 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,628 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,630 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,632 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,633 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,635 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,637 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,639 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,641 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,642 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,644 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,646 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,647 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,649 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,651 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,652 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,654 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,656 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,658 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,659 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,661 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_ar_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,663 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,665 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,667 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,668 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,670 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,672 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,674 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,675 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,677 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,679 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,681 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,682 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,684 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,686 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,688 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,689 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T14:29:35,693 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,694 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,696 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,698 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,700 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,702 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,703 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,705 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,707 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,709 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,710 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,712 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,714 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,716 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,717 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,719 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,720 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,722 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,724 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,725 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,727 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,729 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,730 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,732 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,734 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,735 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,737 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,739 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,741 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,742 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,744 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,746 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,748 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,749 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,751 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,753 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,755 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,756 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,758 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,759 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,761 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,763 copying build/lib/lm_eval/tasks/global_mmlu/full/es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,765 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,768 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,770 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,771 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,773 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,775 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,777 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,778 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,780 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,782 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,784 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,785 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,787 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,789 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,791 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,793 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,795 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_es_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,796 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,798 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,799 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,801 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,803 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,804 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T14:29:35,807 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,808 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,810 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,812 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,814 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,815 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,817 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,819 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,820 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,822 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,824 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,826 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,828 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,829 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,832 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,833 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,835 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,837 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,839 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,840 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,842 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,844 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,845 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,847 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,849 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,851 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,853 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,854 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,856 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,858 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,860 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,862 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,863 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,865 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,867 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,869 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,871 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,872 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,874 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,876 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,878 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,879 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,881 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,883 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,885 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,887 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,888 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,890 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,892 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,894 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,895 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,897 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,899 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,900 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,902 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_pt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,904 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,906 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,908 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,910 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,911 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,913 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,914 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,917 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,918 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,920 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T14:29:35,923 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,924 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,926 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,928 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,930 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,931 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,933 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,935 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,937 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,938 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,940 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,942 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,944 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,945 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,947 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,949 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,951 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,953 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,954 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,956 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,958 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,960 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,961 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,963 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,965 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,967 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,969 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,971 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,972 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,974 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,976 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,978 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,980 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,982 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,983 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,985 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,987 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,988 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,990 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,992 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,993 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,995 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,997 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:35,998 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,000 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,002 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,004 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,006 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,008 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_fil_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,009 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,011 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,013 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,015 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,017 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,018 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,020 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,022 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,024 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,026 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,028 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,029 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,031 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,033 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,035 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,037 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T14:29:36,040 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,041 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,043 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,045 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,047 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,049 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,050 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,052 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,054 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,056 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,058 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,061 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,063 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,065 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,067 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,069 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,070 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,072 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,074 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,076 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,077 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,079 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,081 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,083 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,085 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,087 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,089 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,092 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,094 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,095 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,098 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,100 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,102 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,103 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,105 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,107 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,109 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,111 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,112 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,114 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,116 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,118 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,119 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,121 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,123 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,125 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,127 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,129 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,131 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,133 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,135 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,137 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,139 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,141 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_lt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,143 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,145 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,147 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,149 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,151 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,153 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,155 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,156 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,158 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,160 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,161 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T14:29:36,165 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,166 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,168 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,169 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,171 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,173 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,175 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,176 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,178 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,180 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,183 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,184 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,186 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,188 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,190 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,191 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,193 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,195 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,196 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,198 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,200 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,202 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,204 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,206 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,208 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,210 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_cs_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,212 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,213 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,215 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,217 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,219 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,221 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,222 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,224 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,226 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,228 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,230 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,231 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,233 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,235 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,237 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,239 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,241 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,242 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,244 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,246 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,248 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,250 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,252 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,254 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,257 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,258 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,260 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,262 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,264 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,266 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,267 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,269 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,271 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,273 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,275 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,276 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,278 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,280 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,282 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T14:29:36,286 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,287 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,289 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,291 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,293 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,295 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,297 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,298 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,300 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,302 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,304 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,306 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,308 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,310 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,312 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,313 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,315 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,317 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,318 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,320 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,322 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,323 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,325 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,327 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,329 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_ro_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,330 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,332 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,334 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,335 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,337 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,339 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,341 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,343 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,345 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,347 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,349 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,351 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,353 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,355 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,357 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,359 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,361 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,363 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,365 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,368 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,370 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,372 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,374 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,376 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,379 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,381 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,383 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,385 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,387 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,389 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,391 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,394 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,396 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,398 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,400 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,402 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,404 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,406 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,408 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,410 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T14:29:36,414 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,415 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,418 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,420 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,422 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,424 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,426 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,429 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,431 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,433 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,435 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,437 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,439 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,441 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,444 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,445 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,447 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,449 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,450 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,452 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,453 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,455 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,457 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,459 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,461 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,463 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,465 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,466 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,468 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,470 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,472 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,473 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,475 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,477 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,479 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_ru_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,480 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,482 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,484 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,485 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,487 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,489 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,491 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,492 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,494 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,496 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,497 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,500 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,504 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,506 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,508 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,509 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,511 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,513 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,514 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,516 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,518 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,520 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,521 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,523 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,525 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,527 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,528 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,530 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,532 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,533 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T14:29:36,537 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,538 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,541 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_sr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,543 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,545 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,547 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,548 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,550 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,552 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,553 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,555 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,557 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,559 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,560 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,562 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,564 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,565 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,567 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,569 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,570 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,572 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,574 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,575 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,577 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,579 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,580 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,582 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,584 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,586 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,587 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,589 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,591 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,592 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,594 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,596 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,598 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,600 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,601 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,603 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,604 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,606 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,608 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,609 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,611 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,613 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,614 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,616 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,618 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,620 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,621 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,623 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,625 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,626 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,628 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,630 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,632 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,633 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,635 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,637 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,639 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,640 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,642 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,644 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,645 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,647 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T14:29:36,651 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,652 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,654 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,655 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,657 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,659 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,660 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,662 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,664 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,666 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,667 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,669 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,671 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,673 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,674 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,676 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,678 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,680 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,681 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,683 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,685 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,687 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,689 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,691 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,693 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_el_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,695 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,696 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,699 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,700 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,702 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,704 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,707 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,709 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,711 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,712 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,714 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,716 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,718 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,719 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,721 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,723 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,725 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,727 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,728 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,730 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,732 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,733 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,735 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,737 copying build/lib/lm_eval/tasks/global_mmlu/full/el/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,739 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,741 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,743 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,744 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,746 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,748 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,749 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,751 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,753 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,755 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,756 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,758 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,760 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,761 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,763 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,765 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T14:29:36,768 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,769 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,771 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,773 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,775 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,777 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,779 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,781 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,782 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,784 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,786 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,788 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,789 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,792 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,793 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,795 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,797 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,799 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,800 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,802 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,804 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,805 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,807 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,809 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,810 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,812 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,814 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,816 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,817 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,819 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,821 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,823 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,824 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,826 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,828 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,830 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,832 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,833 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,835 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,837 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,838 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,840 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_zh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,842 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,843 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,845 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,847 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,849 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,851 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,852 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,854 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,856 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,858 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,859 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,861 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,863 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,865 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,867 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,868 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,870 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,872 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,874 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,876 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,878 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,879 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,881 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T14:29:36,884 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,885 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,887 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,889 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,891 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,893 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,894 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,896 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,898 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,899 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,901 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,903 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,905 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,907 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,908 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,910 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,912 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,914 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,916 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,917 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,919 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,921 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,923 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,924 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,926 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,928 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,929 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,931 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,933 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,935 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,936 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,938 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,940 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,942 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,943 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,945 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,947 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,949 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,950 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_mg_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,952 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,954 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,955 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,957 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,959 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,960 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,962 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,964 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,966 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,967 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,969 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,971 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,973 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,974 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,976 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,978 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,979 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,981 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,983 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,985 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,986 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,988 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,990 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,991 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,993 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,994 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T14:29:36,997 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:36,998 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,000 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,002 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,004 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,005 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,007 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,009 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,011 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,012 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,014 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,016 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,017 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,019 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,021 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,023 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,024 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,026 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,028 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,029 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,031 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,032 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,034 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,035 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,037 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,039 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,041 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,042 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,044 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,046 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,048 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,049 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,051 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,053 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,054 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,056 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,058 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,059 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,061 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,063 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,064 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,066 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,067 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,069 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,071 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,072 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,074 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,076 copying build/lib/lm_eval/tasks/global_mmlu/full/id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,077 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,079 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,081 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,083 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,084 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,086 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,088 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,089 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,091 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,093 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,095 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,096 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_id_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,098 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,100 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,101 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,103 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,104 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T14:29:37,108 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,109 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,111 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,112 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,114 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,116 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,118 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,120 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,121 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,123 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,125 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,127 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,129 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,130 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,132 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,134 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,136 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,137 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,139 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,141 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,143 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,144 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,146 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,148 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,150 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,152 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,154 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,155 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,158 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,159 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,161 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,163 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,165 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,166 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,168 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,170 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,172 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,173 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,175 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,177 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,178 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,180 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,182 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,184 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,185 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,187 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_ha_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,189 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,191 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,192 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,194 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,196 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,198 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,200 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,201 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,203 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,205 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,207 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,209 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,211 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,213 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,214 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,216 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,218 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,219 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,221 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T14:29:37,224 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,225 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,227 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,229 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,231 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,233 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,235 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,236 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,238 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,240 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,242 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,243 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,245 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,247 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,249 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,250 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,252 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,254 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,255 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,257 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,259 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,261 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,263 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,264 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,266 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,268 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,270 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,272 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,273 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,275 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_ko_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,277 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,279 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,281 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,282 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,284 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,286 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,288 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,290 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,292 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,294 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,296 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,298 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,300 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,302 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,304 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,306 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,308 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,310 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,312 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,314 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,316 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,317 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,319 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,321 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,323 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,325 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,327 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,329 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,331 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,333 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,335 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,337 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,339 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,340 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,342 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T14:29:37,345 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,346 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,348 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,350 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,352 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,354 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,355 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,357 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,359 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,361 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,362 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,364 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,366 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,368 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,369 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,371 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,373 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,375 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,377 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,378 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,380 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,382 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,383 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,385 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,386 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_ja_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,388 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,390 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,392 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,393 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,395 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,397 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,399 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,400 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,402 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,404 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,406 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,408 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,409 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,411 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,413 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,414 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,416 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,417 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,419 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,421 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,422 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,424 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,426 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,427 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,429 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,431 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,433 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,434 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,437 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,438 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,440 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,442 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,444 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,446 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,448 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,450 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,452 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,454 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,455 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,458 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T14:29:37,461 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,463 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,465 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,467 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,469 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,471 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,474 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,476 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,478 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,480 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,482 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,484 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,487 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,489 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,491 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_pl_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,493 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,495 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,497 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,499 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,500 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,502 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,504 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,506 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,507 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,509 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,511 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,512 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,514 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,516 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,517 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,519 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,521 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,523 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,524 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,527 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,529 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,530 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,532 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,534 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,536 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,538 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,539 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,541 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,543 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,544 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,546 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,548 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,550 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,552 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,555 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,556 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,558 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,560 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,562 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,564 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,565 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,567 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,569 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,571 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,574 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,576 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,578 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,579 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,581 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,583 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T14:29:37,586 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,587 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,589 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,591 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,593 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,594 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,596 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,598 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,600 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,602 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,604 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,605 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,607 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,609 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,611 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,613 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,614 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,616 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,618 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,620 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,621 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,623 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,625 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,626 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,628 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,630 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,632 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,633 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,635 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,637 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,639 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,640 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,642 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,644 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,647 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,649 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,651 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,653 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,656 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,658 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,659 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,661 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,662 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,664 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,666 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,668 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,669 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,671 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,673 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,675 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,677 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,679 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,681 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,684 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,687 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,689 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,691 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,693 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,695 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,696 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,698 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,700 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,702 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_uk_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,704 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,706 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T14:29:37,709 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,710 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,712 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,714 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,715 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,717 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_ig_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,719 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,721 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,723 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,724 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,726 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,728 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,730 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,731 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,733 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,735 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,736 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,738 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,740 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,742 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,744 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,746 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,748 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,750 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,752 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,753 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,755 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,757 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,759 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,761 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,762 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,764 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,766 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,768 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,769 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,771 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,773 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,776 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,779 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,781 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,782 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,784 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,786 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,787 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,789 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,791 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,792 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,794 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,796 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,798 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,799 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,801 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,803 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,805 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,806 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,808 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,810 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,812 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,813 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,815 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,817 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,819 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,821 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,823 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,824 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T14:29:37,828 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,829 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,831 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,832 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,834 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,836 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,838 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,840 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,842 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,843 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,847 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,851 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,853 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,856 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,858 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,861 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,864 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,866 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,868 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,870 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,873 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,875 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,877 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,879 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,881 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,883 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,885 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,888 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,890 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,892 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,894 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,897 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,899 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,901 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,903 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_sw_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,905 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,908 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,910 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,912 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,914 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,916 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,918 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,920 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,922 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,924 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,926 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,928 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,930 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,932 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,934 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,937 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,939 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,941 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,943 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,945 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,947 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,949 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,952 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,954 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,956 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,957 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,959 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,961 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,962 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,964 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T14:29:37,967 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,968 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,970 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,972 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,973 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,975 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,977 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,979 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,980 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,982 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,984 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,985 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,987 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,989 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,990 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,992 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,994 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,996 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,997 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:37,999 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,001 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_sn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,002 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,004 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,006 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,008 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,009 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,011 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,013 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,014 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,016 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,018 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,020 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,021 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,023 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,025 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,026 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,028 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,030 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,032 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,034 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,035 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,037 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,039 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,041 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,042 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,044 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,046 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,047 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,049 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,051 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,053 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,055 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,057 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,059 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,060 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,062 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,064 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,066 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,068 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,070 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,071 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,073 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,075 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,077 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,079 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T14:29:38,082 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,083 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,085 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,087 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,089 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,090 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,092 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,094 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,096 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,098 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,100 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,101 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,103 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,105 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,107 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,108 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,110 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,112 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,114 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,115 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,117 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,119 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,121 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,122 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,124 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,126 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,127 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,129 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,131 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,132 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,134 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,136 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_fr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,138 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,139 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,141 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,143 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,145 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,147 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,148 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,150 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,152 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,154 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,155 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,157 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,159 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,160 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,162 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,164 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,166 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,167 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,169 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,171 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,172 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,174 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,176 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,178 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,179 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,181 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,183 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,185 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,187 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,188 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,190 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,192 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,193 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T14:29:38,197 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,198 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,199 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,201 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,202 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,206 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,208 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,210 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,212 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,214 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,215 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,218 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,219 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,221 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,223 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,225 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,227 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,229 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,231 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,233 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,235 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,236 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,238 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,240 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,242 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,244 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,246 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,248 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,250 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,252 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,254 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,256 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,258 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,260 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,261 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,263 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,265 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,267 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,269 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,271 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,273 copying build/lib/lm_eval/tasks/global_mmlu/full/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,274 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,276 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,278 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,280 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,282 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,284 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,285 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,287 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,289 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,290 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,292 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,294 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,296 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,298 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,300 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,301 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,303 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,305 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,307 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,308 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,310 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,312 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,314 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,315 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T14:29:38,318 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,319 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,321 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,323 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,324 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,326 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,328 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,329 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,331 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,333 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,335 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,336 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,338 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,340 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,342 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,343 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,345 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,347 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_ne_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,349 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,351 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,353 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,354 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,356 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,357 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,359 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,360 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,362 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,364 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,365 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,367 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,369 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,371 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,373 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,374 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,376 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,378 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,379 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,381 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,383 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,385 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,386 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,388 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,390 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,391 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,393 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,395 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,396 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,398 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,400 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,401 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,403 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,405 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,407 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,408 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,410 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,412 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,414 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,415 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,417 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,419 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,421 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,423 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,424 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,426 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,428 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T14:29:38,431 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,432 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,434 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,436 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,438 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,440 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,443 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,445 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,447 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,449 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,451 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,453 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,455 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,457 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,458 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,460 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,462 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,464 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,466 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,467 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,469 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,471 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_am_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,472 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,474 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,476 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,477 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,479 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,481 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,482 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,484 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,486 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,488 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,489 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,491 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,493 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,494 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,496 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,498 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,500 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,502 copying build/lib/lm_eval/tasks/global_mmlu/full/am/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,503 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,505 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,507 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,508 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,510 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,512 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,513 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,515 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,517 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,519 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,520 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,522 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,524 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,525 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,527 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,529 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,531 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,532 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,534 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,536 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,538 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,539 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,541 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,543 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,545 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T14:29:38,548 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,549 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,551 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,552 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,554 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,556 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,557 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,559 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,561 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,563 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,564 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,566 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,568 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,569 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,571 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,573 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,575 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,576 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,578 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,580 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,582 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,583 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,585 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,587 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,588 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,590 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,591 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,593 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,595 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,597 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,598 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,600 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,602 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,603 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,605 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,607 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,608 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,610 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,613 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,617 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,620 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,623 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,625 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,628 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,631 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,635 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,639 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,640 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_yo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,642 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,644 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,645 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,647 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,649 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,650 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,652 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,654 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,655 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,657 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,659 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,660 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,662 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,664 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,665 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,667 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,669 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T14:29:38,672 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,673 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,675 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,676 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,678 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,680 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,681 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,684 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,685 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,687 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,689 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,690 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,692 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,694 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,695 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,697 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,699 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,700 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,702 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,704 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,706 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,707 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,709 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,711 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,712 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,714 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,715 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,717 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,719 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,721 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,722 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,724 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,726 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,728 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,729 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,731 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,733 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,735 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,736 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_te_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,738 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,740 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,741 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,743 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,745 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,747 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,749 copying build/lib/lm_eval/tasks/global_mmlu/full/te/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,750 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,752 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,753 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,755 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,757 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,759 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,761 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,762 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,764 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,766 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,768 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,769 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,771 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,773 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,775 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,777 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,778 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,780 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,782 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T14:29:38,785 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,786 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,788 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,790 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,791 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,793 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,795 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,796 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,798 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,800 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,801 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,803 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,805 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,807 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,809 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,810 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,812 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,814 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,816 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,817 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,819 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,821 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,822 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,824 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,826 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,827 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,829 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,831 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,832 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,834 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,836 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,837 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,839 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,841 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,842 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,844 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,846 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,848 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,849 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,851 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,853 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,855 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,856 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,858 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,860 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,861 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,863 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,865 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,866 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,868 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,870 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,872 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,873 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,875 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,877 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,879 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,880 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_ms_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,882 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,884 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,886 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,887 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,889 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,891 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,893 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,895 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T14:29:38,898 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,899 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,901 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,902 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,904 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,905 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,907 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,909 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,910 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,912 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,914 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,916 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,917 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,919 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,921 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,923 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,924 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,926 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,928 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,930 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,932 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,933 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,935 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,937 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,939 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,940 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,942 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,943 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,945 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,947 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,949 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,951 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,952 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,954 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,956 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,957 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_he_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,959 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,961 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,963 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,965 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,967 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,969 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,970 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,972 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,974 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,975 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,977 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,979 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,980 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,982 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,984 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,985 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,987 copying build/lib/lm_eval/tasks/global_mmlu/full/he/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,989 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,990 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,992 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,994 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,995 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,997 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:38,999 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:39,001 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:39,002 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:39,004 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:39,006 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:39,007 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T14:29:39,011 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,012 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,014 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,015 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,017 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,019 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,021 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,023 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,025 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,027 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,028 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,030 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,033 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,034 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,036 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,039 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,040 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,042 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,044 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,046 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,048 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,049 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,051 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,053 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,054 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,056 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,058 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,059 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,061 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_sv_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,063 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,064 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,066 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,068 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,069 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,071 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,073 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,074 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,076 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,078 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,080 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,082 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,083 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,085 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,087 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,089 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,091 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,093 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,095 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,096 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,098 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,100 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,103 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,105 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,107 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,109 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,111 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,113 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,115 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,117 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,119 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,121 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,123 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,125 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,126 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,128 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T14:29:39,132 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,133 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,135 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,137 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,139 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,140 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,142 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,144 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,146 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,148 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,151 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,153 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,155 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,157 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,159 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,161 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,163 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,165 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,167 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,169 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,171 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,173 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,174 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,176 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,178 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,179 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,181 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,183 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,184 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,186 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,188 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,190 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,191 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,193 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,195 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,197 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,199 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,200 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,202 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,204 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,206 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,208 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,210 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,212 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,213 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,215 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,217 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,218 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,220 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_vi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,222 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,223 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,225 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,227 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,229 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,231 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,232 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,234 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,236 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,238 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,239 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,241 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,243 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,245 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,247 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,249 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T14:29:39,252 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,253 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,255 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,256 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,258 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,260 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,261 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,263 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,265 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,267 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,268 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,270 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,272 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,274 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,275 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,277 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,279 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,281 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,282 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,284 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,286 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,288 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,289 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,291 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,293 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,294 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,296 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,298 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,299 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,301 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,303 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,305 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,306 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,308 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,310 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,311 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,313 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,316 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,319 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,321 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,325 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,328 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,330 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,332 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,334 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,336 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,337 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,339 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,341 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,343 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_ky_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,344 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,346 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,348 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,349 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,351 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,353 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,355 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,357 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,359 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,360 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,362 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,364 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,367 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,369 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,371 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T14:29:39,375 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,376 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,379 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,381 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,383 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,385 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,387 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,389 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,391 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,393 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,395 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,398 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,400 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,402 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,404 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,407 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,409 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,411 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,413 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,415 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,418 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,420 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,422 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,424 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,426 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,429 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,431 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,433 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,435 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,437 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,439 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,441 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,444 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,446 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,448 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,450 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,453 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,455 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,458 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,460 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,462 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,464 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,466 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_fa_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,468 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,470 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,472 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,473 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,475 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,477 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,478 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,480 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,482 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,483 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,485 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,487 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,488 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,490 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,492 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,494 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,495 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,497 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,499 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,501 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,502 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,504 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T14:29:39,507 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,508 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,510 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,512 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,514 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,515 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,517 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,519 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,521 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,522 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,524 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,526 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,527 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,529 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,531 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,533 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,535 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,537 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,538 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,540 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,542 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,544 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,546 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,547 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,549 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,551 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,553 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,554 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,556 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,558 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,560 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,561 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,563 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,565 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,567 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,569 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,571 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,573 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,575 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,577 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,579 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,581 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,583 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,585 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,587 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,589 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,590 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,592 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,594 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,595 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,597 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,599 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,600 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,602 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,604 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,606 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,608 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,609 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,611 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,613 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,614 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,616 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_nl_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,618 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,620 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,621 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T14:29:39,625 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,625 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,628 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,629 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,631 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,633 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,634 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,636 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,638 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,639 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,641 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,643 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,645 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,647 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,648 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,650 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,652 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,654 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,656 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,657 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,659 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,661 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,663 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,665 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,666 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,668 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,670 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,671 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,673 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,675 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,677 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,678 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,680 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,682 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,683 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,685 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,687 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,689 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_ny_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,691 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,693 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,694 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,696 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,698 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,700 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,702 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,703 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,705 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,707 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,709 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,711 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,712 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,714 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,716 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,718 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,719 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,721 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,723 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,724 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,726 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,728 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,730 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,731 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,733 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,735 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,737 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T14:29:39,740 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,741 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,743 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,744 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,746 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,748 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,750 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,752 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,753 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,755 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,757 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,759 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,761 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,762 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,764 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,766 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,768 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,769 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_tr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,771 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,773 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,774 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,776 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,778 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,780 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,781 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,783 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,785 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,786 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,788 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,790 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,792 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,794 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,795 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,797 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,799 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,801 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,802 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,804 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,806 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,808 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,810 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,811 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,813 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,815 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,817 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,819 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,820 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,822 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,824 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,825 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,827 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,829 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,830 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,832 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,834 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,836 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,838 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,839 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,841 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,843 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,845 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,846 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,848 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,850 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,852 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T14:29:39,855 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,856 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,858 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,860 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,862 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,863 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,865 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,867 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,869 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,871 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,873 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,875 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,877 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,879 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,881 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,884 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,886 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,888 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,891 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,893 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,895 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,897 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,899 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,901 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,903 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,905 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,906 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,908 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,910 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,912 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,914 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,916 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,918 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,919 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,921 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,923 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,925 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,926 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,928 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,930 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,932 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,933 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,935 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,937 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,939 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,941 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,944 copying build/lib/lm_eval/tasks/global_mmlu/full/it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,946 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,948 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,949 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,951 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,953 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,955 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,957 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_it_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,959 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,961 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,962 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,964 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,966 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,968 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,969 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,971 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,973 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,975 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,977 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T14:29:39,980 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,981 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,983 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,985 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,986 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,988 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,989 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,991 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,993 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,994 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,996 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:39,998 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,000 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,001 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,003 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,005 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,007 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,009 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,010 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,012 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,014 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,016 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,017 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,019 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,021 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,023 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,025 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,026 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_so_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,028 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,030 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,031 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,033 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,035 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,036 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,038 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,040 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,042 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,043 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,045 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,047 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,049 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,051 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,052 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,054 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,056 copying build/lib/lm_eval/tasks/global_mmlu/full/so/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,058 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,059 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,061 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,063 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,064 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,066 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,068 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,069 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,071 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,073 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,075 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,076 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,078 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,080 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,082 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,083 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,085 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,087 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,089 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,090 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T14:29:40,093 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,095 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,097 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,098 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,100 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,102 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,103 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,105 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,106 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,108 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,110 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,111 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,113 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,115 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,117 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,118 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,120 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,122 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,124 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,125 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,127 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,129 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,131 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,133 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,134 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,136 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,138 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,139 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,141 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,142 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_si_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,144 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,146 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,148 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,149 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,151 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,153 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,154 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,156 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,158 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,160 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,161 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,163 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,165 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,166 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,168 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,170 copying build/lib/lm_eval/tasks/global_mmlu/full/si/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,172 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,174 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,175 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,177 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,179 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,181 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,182 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,184 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,186 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,188 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,189 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,191 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,193 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,194 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,196 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,198 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,200 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,201 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,203 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T14:29:40,205 copying build/lib/lm_eval/tasks/global_mmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu 2026-05-11T14:29:40,208 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default 2026-05-11T14:29:40,210 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,211 copying build/lib/lm_eval/tasks/global_mmlu/default/de/_global_mmlu_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,212 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,214 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,216 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,218 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,220 copying build/lib/lm_eval/tasks/global_mmlu/default/de/_de_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,221 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,223 copying build/lib/lm_eval/tasks/global_mmlu/default/de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,225 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T14:29:40,227 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,228 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,230 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,232 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,233 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,235 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/_global_mmlu_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,237 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,238 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,240 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/_hi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,242 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T14:29:40,244 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,245 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,247 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,249 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/_bn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,250 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,252 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,254 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/_global_mmlu_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,255 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,257 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,259 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T14:29:40,261 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,262 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,264 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,266 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,268 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,269 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,271 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,273 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/_ar_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,274 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/_global_mmlu_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,276 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T14:29:40,279 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,280 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,282 copying build/lib/lm_eval/tasks/global_mmlu/default/es/_global_mmlu_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,284 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,285 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,287 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,289 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,291 copying build/lib/lm_eval/tasks/global_mmlu/default/es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,293 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,294 copying build/lib/lm_eval/tasks/global_mmlu/default/es/_es_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T14:29:40,297 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,298 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,300 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,302 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,303 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,305 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,306 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,308 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,310 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,311 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T14:29:40,314 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,315 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,317 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,318 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,320 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,322 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,324 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,326 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,327 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,329 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T14:29:40,331 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,332 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,334 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,336 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,338 copying build/lib/lm_eval/tasks/global_mmlu/default/id/_global_mmlu_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,339 copying build/lib/lm_eval/tasks/global_mmlu/default/id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,341 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,343 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,345 copying build/lib/lm_eval/tasks/global_mmlu/default/id/_id_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,346 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T14:29:40,348 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,349 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,351 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,353 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,355 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,357 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,358 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,360 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,362 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,364 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T14:29:40,366 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,367 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,369 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/_ja_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,370 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,372 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,374 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,376 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/_global_mmlu_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,377 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,379 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,381 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T14:29:40,383 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,384 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,386 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/_sw_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,387 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,389 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,391 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,392 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,394 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,396 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/_global_mmlu_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,398 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T14:29:40,400 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,401 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/_global_mmlu_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,403 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,405 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,406 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/_fr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,408 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,410 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,412 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,413 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,415 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T14:29:40,418 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,419 copying build/lib/lm_eval/tasks/global_mmlu/default/en/_global_mmlu_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,420 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,422 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,424 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,425 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,427 copying build/lib/lm_eval/tasks/global_mmlu/default/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,429 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,430 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,432 copying build/lib/lm_eval/tasks/global_mmlu/default/en/_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T14:29:40,434 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,435 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,437 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,439 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,441 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/_global_mmlu_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,443 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,445 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,447 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/_yo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,449 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,451 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T14:29:40,454 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,455 copying build/lib/lm_eval/tasks/global_mmlu/default/it/_global_mmlu_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,458 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,460 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,461 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,463 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,465 copying build/lib/lm_eval/tasks/global_mmlu/default/it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,466 copying build/lib/lm_eval/tasks/global_mmlu/default/it/_it_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,468 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,470 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T14:29:40,473 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mlqa 2026-05-11T14:29:40,474 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,476 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,478 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,480 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,482 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,483 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,485 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,487 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,489 copying build/lib/lm_eval/tasks/mlqa/generate_tasks.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,491 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,492 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,494 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,496 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,497 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,499 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,501 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,503 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,504 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,506 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,508 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,510 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,511 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,513 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,515 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,517 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,518 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,520 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,522 copying build/lib/lm_eval/tasks/mlqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,524 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,525 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,527 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,529 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,530 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,532 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,534 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,536 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,537 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,539 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,541 copying build/lib/lm_eval/tasks/mlqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,543 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,545 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,546 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,548 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,550 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,552 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,554 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,555 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,557 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,559 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,561 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,563 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,565 copying build/lib/lm_eval/tasks/mlqa/mlqa_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,566 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T14:29:40,569 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,570 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,571 copying build/lib/lm_eval/tasks/japanese_leaderboard/requirements.txt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,573 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jsquad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,575 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,576 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jaqket_v2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,578 copying build/lib/lm_eval/tasks/japanese_leaderboard/_ja_leaderboard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,580 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,581 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,583 copying build/lib/lm_eval/tasks/japanese_leaderboard/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,585 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,587 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,589 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,591 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,592 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_marc_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,594 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T14:29:40,596 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa 2026-05-11T14:29:40,598 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2026-05-11T14:29:40,599 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T14:29:40,601 copying build/lib/lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T14:29:40,603 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T14:29:40,604 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T14:29:40,606 copying build/lib/lm_eval/tasks/gpqa/generative/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T14:29:40,608 copying build/lib/lm_eval/tasks/gpqa/generative/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T14:29:40,609 copying build/lib/lm_eval/tasks/gpqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa 2026-05-11T14:29:40,612 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:40,613 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:40,614 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:40,616 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:40,618 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:40,620 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:40,621 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T14:29:40,623 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:40,624 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:40,626 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:40,628 copying build/lib/lm_eval/tasks/gpqa/n_shot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:40,630 copying build/lib/lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:40,631 copying build/lib/lm_eval/tasks/gpqa/n_shot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:40,633 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T14:29:40,635 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:40,636 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:40,638 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:40,640 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:40,642 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:40,643 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:40,645 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T14:29:40,648 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:40,649 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:40,650 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:40,652 copying build/lib/lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:40,654 copying build/lib/lm_eval/tasks/gpqa/zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:40,656 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:40,658 copying build/lib/lm_eval/tasks/gpqa/zeroshot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T14:29:40,660 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/portuguese_bench 2026-05-11T14:29:40,661 copying build/lib/lm_eval/tasks/portuguese_bench/assin_entailment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T14:29:40,663 copying build/lib/lm_eval/tasks/portuguese_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T14:29:40,665 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,666 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,668 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_gl-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,669 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_fr-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,671 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,673 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,675 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_it-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,677 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_en-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,679 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,680 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,682 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_es-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,684 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_de-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,686 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,688 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_ca-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,690 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,692 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/create_yamls_flores_pt.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,694 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_eu-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,695 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,697 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,699 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T14:29:40,701 copying build/lib/lm_eval/tasks/portuguese_bench/portuguese_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T14:29:40,703 copying build/lib/lm_eval/tasks/portuguese_bench/assin_paraphrase.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T14:29:40,705 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ruler 2026-05-11T14:29:40,706 copying build/lib/lm_eval/tasks/ruler/common_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,708 copying build/lib/lm_eval/tasks/ruler/fwe_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,710 copying build/lib/lm_eval/tasks/ruler/niah_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,712 copying build/lib/lm_eval/tasks/ruler/fwe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,714 copying build/lib/lm_eval/tasks/ruler/qa_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,717 copying build/lib/lm_eval/tasks/ruler/vt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,718 copying build/lib/lm_eval/tasks/ruler/qa_hotpot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,720 copying build/lib/lm_eval/tasks/ruler/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,722 copying build/lib/lm_eval/tasks/ruler/niah_single_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,723 copying build/lib/lm_eval/tasks/ruler/niah_multikey_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,725 copying build/lib/lm_eval/tasks/ruler/niah_single_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,727 copying build/lib/lm_eval/tasks/ruler/prepare_niah.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,729 copying build/lib/lm_eval/tasks/ruler/vt_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,732 copying build/lib/lm_eval/tasks/ruler/niah_single_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,733 copying build/lib/lm_eval/tasks/ruler/qa_squad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,735 copying build/lib/lm_eval/tasks/ruler/cwe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,737 copying build/lib/lm_eval/tasks/ruler/ruler.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,739 copying build/lib/lm_eval/tasks/ruler/niah_multikey_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,741 copying build/lib/lm_eval/tasks/ruler/essays.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,743 copying build/lib/lm_eval/tasks/ruler/niah_multiquery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,745 copying build/lib/lm_eval/tasks/ruler/niah_multikey_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,747 copying build/lib/lm_eval/tasks/ruler/cwe_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,749 copying build/lib/lm_eval/tasks/ruler/niah_multivalue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T14:29:40,752 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox 2026-05-11T14:29:40,754 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,755 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,757 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,759 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,762 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,764 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,765 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,767 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,769 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,771 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,773 copying build/lib/lm_eval/tasks/mmlu_prox/de/_de_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,775 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,776 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,779 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,780 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,783 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,784 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,787 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,789 copying build/lib/lm_eval/tasks/mmlu_prox/de/_mmlu_prox_lite_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,791 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,793 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,795 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,797 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,799 copying build/lib/lm_eval/tasks/mmlu_prox/de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,801 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,803 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,806 copying build/lib/lm_eval/tasks/mmlu_prox/de/_mmlu_prox_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,809 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,812 copying build/lib/lm_eval/tasks/mmlu_prox/de/_de_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,816 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,819 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,820 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,822 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,823 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T14:29:40,826 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,827 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,829 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,830 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_hi_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,832 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,834 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_lite_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,836 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,837 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,839 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,841 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,843 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,845 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,847 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,849 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,850 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,852 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,854 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,855 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,857 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,859 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,861 copying build/lib/lm_eval/tasks/mmlu_prox/hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,863 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,865 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,866 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,868 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,870 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,871 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,873 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,875 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,877 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,878 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,880 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,882 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_hi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,883 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T14:29:40,886 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,887 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,889 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,891 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,892 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,894 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,896 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,898 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,899 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,901 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,903 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_lite_ur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,905 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,907 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_ur_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,909 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,910 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,912 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,914 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,915 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,917 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,920 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_ur_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,923 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,925 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,927 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,929 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,930 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,932 copying build/lib/lm_eval/tasks/mmlu_prox/ur/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,934 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,936 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,937 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,939 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,941 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,943 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_ur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,944 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,946 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T14:29:40,949 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,950 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,952 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,954 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,956 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,958 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,960 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,961 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,963 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,965 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,967 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,969 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,971 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,973 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,975 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,977 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,979 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,980 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,982 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,984 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,986 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_bn_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,988 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_lite_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,990 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_bn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,992 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,993 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,995 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,997 copying build/lib/lm_eval/tasks/mmlu_prox/bn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:40,999 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:41,001 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:41,003 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:41,005 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:41,006 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:41,008 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:41,010 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T14:29:41,014 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,015 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_lite_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,017 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,019 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,021 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_ar_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,023 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,025 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,027 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,029 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,031 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,034 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,036 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,038 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,040 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,041 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,043 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,045 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,047 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,049 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,050 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,052 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,054 copying build/lib/lm_eval/tasks/mmlu_prox/ar/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,056 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,058 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,059 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_ar_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,061 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,063 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,065 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,067 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,068 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,071 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,072 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,074 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,076 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T14:29:41,079 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,080 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,082 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,083 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,085 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,087 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,089 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,090 copying build/lib/lm_eval/tasks/mmlu_prox/es/_mmlu_prox_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,092 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,094 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,096 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,098 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,099 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,101 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,103 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,104 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,106 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,108 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,109 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,111 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,113 copying build/lib/lm_eval/tasks/mmlu_prox/es/_es_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,115 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,116 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,118 copying build/lib/lm_eval/tasks/mmlu_prox/es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,120 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,122 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,123 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,125 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,127 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,128 copying build/lib/lm_eval/tasks/mmlu_prox/es/_mmlu_prox_lite_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,130 copying build/lib/lm_eval/tasks/mmlu_prox/es/_es_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,132 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,134 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,135 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T14:29:41,139 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,140 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,142 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,144 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,145 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,147 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,149 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,151 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,153 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,154 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,156 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_lite_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,158 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,159 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,161 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,163 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,164 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,166 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,168 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,170 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,172 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,173 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_pt_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,175 copying build/lib/lm_eval/tasks/mmlu_prox/pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,177 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,179 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,180 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_pt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,182 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,184 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,185 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,187 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,189 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,191 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,192 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,194 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,196 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T14:29:41,198 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,199 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,201 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,203 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,204 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,206 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,208 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,210 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,212 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,213 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,215 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,217 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,219 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,221 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,222 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,224 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,226 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_lite_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,228 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,229 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,231 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,233 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,235 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,237 copying build/lib/lm_eval/tasks/mmlu_prox/zu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,238 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,240 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,242 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_zu_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,243 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,245 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,247 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,248 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_zu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,250 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,252 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,253 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,255 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T14:29:41,257 copying build/lib/lm_eval/tasks/mmlu_prox/mmlu_prox_lite_config_generator.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T14:29:41,260 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,261 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,263 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,265 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,267 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,268 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,270 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_cs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,272 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,274 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,276 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,278 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,279 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,281 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_lite_cs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,283 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,285 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,287 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,288 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,290 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_cs_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,292 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,293 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,295 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,297 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,299 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,301 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,302 copying build/lib/lm_eval/tasks/mmlu_prox/cs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,304 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_cs_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,306 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,308 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,310 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,311 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,313 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,315 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,317 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,318 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T14:29:41,321 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,322 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_lite_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,324 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,326 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,328 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,329 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,331 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,333 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,335 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,338 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,340 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,342 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,344 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,346 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,347 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,349 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,351 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,353 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,354 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,356 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,359 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,361 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,363 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mr_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,366 copying build/lib/lm_eval/tasks/mmlu_prox/mr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,368 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,369 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,371 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,373 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,375 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,377 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,379 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,380 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,382 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,384 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T14:29:41,386 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,387 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,389 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,391 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,393 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,394 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,396 copying build/lib/lm_eval/tasks/mmlu_prox/af/_af_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,398 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,400 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,401 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,403 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,405 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,407 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,408 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,410 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,412 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,414 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,415 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,417 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,419 copying build/lib/lm_eval/tasks/mmlu_prox/af/_af_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,421 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,422 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,424 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,426 copying build/lib/lm_eval/tasks/mmlu_prox/af/_mmlu_prox_af.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,427 copying build/lib/lm_eval/tasks/mmlu_prox/af/_mmlu_prox_lite_af.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,429 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,431 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,432 copying build/lib/lm_eval/tasks/mmlu_prox/af/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,434 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,436 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,437 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,439 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,441 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,442 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T14:29:41,445 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,446 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,448 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,450 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,452 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,454 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,455 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,458 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,460 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,463 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,498 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,500 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_lite_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,502 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,504 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,506 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,508 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,510 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,512 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,514 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,516 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,517 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,519 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,521 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,522 copying build/lib/lm_eval/tasks/mmlu_prox/wo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,524 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_wo_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,526 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,528 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_wo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,529 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,531 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,533 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,535 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,536 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,538 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,540 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T14:29:41,542 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,544 copying build/lib/lm_eval/tasks/mmlu_prox/th/_mmlu_prox_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,545 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,547 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,549 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,551 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,552 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,554 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,556 copying build/lib/lm_eval/tasks/mmlu_prox/th/_th_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,557 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,559 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,561 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,562 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,564 copying build/lib/lm_eval/tasks/mmlu_prox/th/_th_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,566 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,568 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,569 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,571 copying build/lib/lm_eval/tasks/mmlu_prox/th/_mmlu_prox_lite_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,573 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,574 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,576 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,578 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,580 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,582 copying build/lib/lm_eval/tasks/mmlu_prox/th/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,584 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,587 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,589 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,591 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,593 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,595 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,597 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,599 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,601 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,603 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T14:29:41,606 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,607 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,609 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,611 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,612 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,614 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,616 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,617 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,619 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,621 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,623 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,624 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,626 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,628 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,630 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,632 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,633 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,635 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,637 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_ru_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,639 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,641 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,642 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,644 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,646 copying build/lib/lm_eval/tasks/mmlu_prox/ru/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,648 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,649 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,651 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,653 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_ru_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,654 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,656 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,658 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_lite_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,659 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,661 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,663 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T14:29:41,665 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,666 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_sr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,668 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,670 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,672 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,673 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,675 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,677 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,678 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_lite_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,680 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,682 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,683 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,685 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,687 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,689 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,690 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,692 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,694 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,696 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_sr_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,698 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,700 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,703 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,705 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,708 copying build/lib/lm_eval/tasks/mmlu_prox/sr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,709 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,711 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,713 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,715 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,716 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,718 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,720 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,721 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,723 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,725 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T14:29:41,728 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,729 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,731 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,733 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,735 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,736 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,738 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,740 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,741 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,743 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,745 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,747 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,748 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,750 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_lite_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,752 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,754 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,755 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,757 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,759 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,761 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,762 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,764 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_zh_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,766 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,768 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_zh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,770 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,771 copying build/lib/lm_eval/tasks/mmlu_prox/zh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,773 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,775 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,777 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,779 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,780 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,782 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,784 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,785 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T14:29:41,788 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,789 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,791 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,793 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,795 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,797 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,798 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,800 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,802 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,803 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,805 copying build/lib/lm_eval/tasks/mmlu_prox/id/_mmlu_prox_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,807 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,809 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,810 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,812 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,814 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,815 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,817 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,819 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,821 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,822 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,824 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,826 copying build/lib/lm_eval/tasks/mmlu_prox/id/_mmlu_prox_lite_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,828 copying build/lib/lm_eval/tasks/mmlu_prox/id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,830 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,831 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,833 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,835 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,836 copying build/lib/lm_eval/tasks/mmlu_prox/id/_id_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,838 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,840 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,841 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,843 copying build/lib/lm_eval/tasks/mmlu_prox/id/_id_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,845 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T14:29:41,847 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,848 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,850 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,852 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,854 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,855 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,857 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,859 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,861 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,862 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_ko_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,864 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_lite_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,866 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,868 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,869 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,871 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,873 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,875 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,876 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_ko_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,878 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,879 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,881 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,884 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,886 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,887 copying build/lib/lm_eval/tasks/mmlu_prox/ko/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,889 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,891 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,893 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,894 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,896 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,898 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,900 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,901 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,903 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,905 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T14:29:41,908 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,909 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,910 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,912 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,914 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,916 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,918 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,919 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_ja_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,921 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,923 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_ja_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,924 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,926 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,927 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,929 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,931 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,932 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,934 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,936 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,938 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,940 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,941 copying build/lib/lm_eval/tasks/mmlu_prox/ja/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,943 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,945 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,946 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,948 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,950 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,951 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,953 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_lite_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,955 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,957 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,959 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,961 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,962 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,964 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T14:29:41,966 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,967 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,969 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,971 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,973 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,975 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,976 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,978 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,980 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,982 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,984 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,985 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,987 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,989 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,991 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,992 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,994 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_uk_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,996 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,998 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:41,999 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,001 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,003 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_lite_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,005 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,007 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,008 copying build/lib/lm_eval/tasks/mmlu_prox/uk/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,010 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,012 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,014 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,015 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,017 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,019 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,020 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,022 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_uk_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,024 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T14:29:42,025 copying build/lib/lm_eval/tasks/mmlu_prox/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T14:29:42,028 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/template 2026-05-11T14:29:42,029 copying build/lib/lm_eval/tasks/mmlu_prox/template/_lang_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/template 2026-05-11T14:29:42,031 copying build/lib/lm_eval/tasks/mmlu_prox/template/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/template 2026-05-11T14:29:42,034 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,035 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,037 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,039 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,041 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,042 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,044 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,046 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,048 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,049 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,051 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_lite_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,053 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,054 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,056 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,058 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,059 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,061 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,063 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_sw_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,064 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,066 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,068 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,070 copying build/lib/lm_eval/tasks/mmlu_prox/sw/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,072 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,074 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,075 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,077 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,079 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,081 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,082 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,084 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,086 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,087 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,089 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,091 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_sw_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T14:29:42,093 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,094 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,097 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,099 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,100 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,102 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,104 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,106 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,108 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,110 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,111 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,113 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,115 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,116 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,118 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_fr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,120 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,122 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_lite_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,124 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,125 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,127 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,129 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,131 copying build/lib/lm_eval/tasks/mmlu_prox/fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,132 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_fr_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,134 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,136 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,138 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,139 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,141 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,143 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,145 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,147 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,149 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,150 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,152 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T14:29:42,155 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,156 copying build/lib/lm_eval/tasks/mmlu_prox/en/_mmlu_prox_lite_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,158 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,159 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,161 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,163 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,164 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,166 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,168 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,170 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,172 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,174 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,175 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,177 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,179 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,181 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,182 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,184 copying build/lib/lm_eval/tasks/mmlu_prox/en/_en_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,186 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,191 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,193 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,195 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,196 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,198 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,200 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,201 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,203 copying build/lib/lm_eval/tasks/mmlu_prox/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,205 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,207 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,209 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,211 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,213 copying build/lib/lm_eval/tasks/mmlu_prox/en/_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,215 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,218 copying build/lib/lm_eval/tasks/mmlu_prox/en/_mmlu_prox_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T14:29:42,221 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,222 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,225 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,227 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,230 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_lite_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,232 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_ne_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,235 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,237 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,239 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,241 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,243 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,245 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,247 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,249 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,251 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,253 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,255 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,257 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,259 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,261 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,263 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_ne_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,265 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,267 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,269 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,271 copying build/lib/lm_eval/tasks/mmlu_prox/ne/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,273 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,275 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,277 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,279 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,280 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,282 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,284 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,286 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,288 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T14:29:42,292 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,293 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,295 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,298 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,300 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,301 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,303 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_lite_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,305 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,306 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,308 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,310 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,311 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,313 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,315 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,317 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,319 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,321 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,323 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,324 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,326 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,328 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,329 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,331 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,333 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_yo_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,335 copying build/lib/lm_eval/tasks/mmlu_prox/yo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,337 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,338 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_yo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,340 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,342 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,344 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,345 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,347 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,348 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,350 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T14:29:42,353 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,354 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,356 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,357 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,359 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,361 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,362 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,364 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,366 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,368 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,370 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,372 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,374 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,377 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,379 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,381 copying build/lib/lm_eval/tasks/mmlu_prox/te/_mmlu_prox_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,383 copying build/lib/lm_eval/tasks/mmlu_prox/te/_te_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,385 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,387 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,389 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,390 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,392 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,394 copying build/lib/lm_eval/tasks/mmlu_prox/te/_mmlu_prox_lite_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,395 copying build/lib/lm_eval/tasks/mmlu_prox/te/_te_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,397 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,399 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,401 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,402 copying build/lib/lm_eval/tasks/mmlu_prox/te/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,404 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,406 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,407 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,409 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,411 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,413 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T14:29:42,415 copying build/lib/lm_eval/tasks/mmlu_prox/lang_libs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T14:29:42,419 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,420 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,421 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,423 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,425 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,427 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,429 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,430 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,432 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_lite_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,434 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,436 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,437 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,439 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,441 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,442 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,444 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,446 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,447 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,449 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,451 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,452 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,454 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,456 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_vi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,458 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,460 copying build/lib/lm_eval/tasks/mmlu_prox/vi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,462 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,463 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,465 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,467 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_vi_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,469 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,470 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,472 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,474 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,476 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T14:29:42,477 copying build/lib/lm_eval/tasks/mmlu_prox/mmlu_prox_config_generator.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T14:29:42,480 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,481 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,483 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,485 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,486 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,488 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,490 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,491 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,493 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,495 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,497 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,498 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,500 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,502 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,504 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_hu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,506 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_lite_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,508 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,510 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,512 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,513 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,515 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,517 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,519 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,521 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,522 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,524 copying build/lib/lm_eval/tasks/mmlu_prox/hu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,526 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,528 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,529 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,531 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,533 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,534 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,536 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_hu_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,538 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T14:29:42,540 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,542 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,543 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,545 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,547 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,549 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,551 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,552 copying build/lib/lm_eval/tasks/mmlu_prox/it/_mmlu_prox_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,554 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,556 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,558 copying build/lib/lm_eval/tasks/mmlu_prox/it/_it_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,559 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,561 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,563 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,565 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,566 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,568 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,570 copying build/lib/lm_eval/tasks/mmlu_prox/it/_mmlu_prox_lite_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,572 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,574 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,575 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,577 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,579 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,580 copying build/lib/lm_eval/tasks/mmlu_prox/it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,582 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,584 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,585 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,587 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,589 copying build/lib/lm_eval/tasks/mmlu_prox/it/_it_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,591 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,592 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,594 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,596 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,598 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T14:29:42,600 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/swde 2026-05-11T14:29:42,601 copying build/lib/lm_eval/tasks/swde/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swde 2026-05-11T14:29:42,603 copying build/lib/lm_eval/tasks/swde/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swde 2026-05-11T14:29:42,605 copying build/lib/lm_eval/tasks/swde/swde.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swde 2026-05-11T14:29:42,607 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu 2026-05-11T14:29:42,609 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,611 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,613 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,614 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,616 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,618 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,620 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,622 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,624 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,625 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,627 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,629 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,630 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,632 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,633 copying build/lib/lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,635 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,637 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,639 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,640 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,642 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,644 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,645 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,647 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,649 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,651 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,652 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,654 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,656 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,658 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,659 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,661 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,663 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,665 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,666 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,668 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,670 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,671 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,673 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,675 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,676 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,678 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,680 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,681 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,683 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,685 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,687 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,689 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,691 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,692 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,694 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,696 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,698 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T14:29:42,701 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,702 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,704 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,705 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,707 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,709 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,710 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,712 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,713 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,715 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,717 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,718 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,720 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,722 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,723 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,725 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,727 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,728 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,730 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,732 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,733 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,735 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,737 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,739 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,740 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,742 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,744 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,745 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,747 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,749 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,750 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,752 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,754 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,755 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,757 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,759 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,760 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,762 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,764 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,765 copying build/lib/lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,767 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,769 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,770 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,772 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,774 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,776 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,777 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,779 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,781 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,782 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,784 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,786 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T14:29:42,788 copying build/lib/lm_eval/tasks/kmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu 2026-05-11T14:29:42,792 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,793 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,796 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,798 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,800 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,803 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,805 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,808 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,810 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,812 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,814 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,816 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,818 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,821 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,823 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,825 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,827 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,829 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,831 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,833 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,836 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,838 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,841 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,843 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,845 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,847 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,850 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,852 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,854 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,856 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,858 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,860 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,863 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,865 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,867 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,869 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,871 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,874 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,876 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,878 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,880 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,882 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,884 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,887 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,889 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,891 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,893 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,896 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,898 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,901 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,903 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,904 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T14:29:42,908 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,909 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,910 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,912 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,914 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,915 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,917 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,919 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,920 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,922 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,924 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,926 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,927 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,929 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,931 copying build/lib/lm_eval/tasks/kmmlu/default/_default_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,933 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,934 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,936 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,938 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,940 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,942 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,943 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,945 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,947 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,948 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,950 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,952 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,953 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,955 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,957 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,958 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,960 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,962 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,964 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,966 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,968 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,970 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,971 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,973 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,975 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,976 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,978 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,980 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,982 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,983 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,985 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,987 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,989 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,990 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,992 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,994 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:42,997 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T14:29:43,001 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,002 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,004 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,006 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,008 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,010 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,013 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,015 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,017 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,019 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,021 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,023 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,025 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,028 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,030 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,032 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,034 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,036 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,039 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,041 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,043 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,046 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,048 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,050 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,052 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,055 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,057 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,060 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,062 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,064 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,066 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,069 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,071 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,073 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,075 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,077 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,080 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,082 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,084 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,086 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,089 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,091 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,093 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,095 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,097 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,098 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,100 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,102 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,104 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,106 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,108 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,109 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T14:29:43,113 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,114 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,116 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_biology_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,117 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_math_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,119 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,121 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_political_science_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,122 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,124 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,126 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,128 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_history_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,129 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_management_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,131 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,133 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_geography_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,135 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_geography_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,137 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,138 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_accounting_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,140 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_economics_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,142 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,143 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,145 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,147 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_civics_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,148 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,150 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,152 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_history_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,154 copying build/lib/lm_eval/tasks/arabicmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,155 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,157 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,159 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,161 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_physics_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,163 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,164 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,166 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,168 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,170 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_economics_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,171 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_civics_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,173 copying build/lib/lm_eval/tasks/arabicmmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,175 copying build/lib/lm_eval/tasks/arabicmmlu/_default_arabicmmlu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,177 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,178 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,180 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_philosophy_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,182 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,183 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,185 copying build/lib/lm_eval/tasks/arabicmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,187 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_geography_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,188 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,190 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_history_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,192 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,193 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_economics_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,195 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,197 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_law_professional.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,199 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T14:29:43,201 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noticia 2026-05-11T14:29:43,202 copying build/lib/lm_eval/tasks/noticia/noticia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noticia 2026-05-11T14:29:43,204 copying build/lib/lm_eval/tasks/noticia/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noticia 2026-05-11T14:29:43,206 copying build/lib/lm_eval/tasks/noticia/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noticia 2026-05-11T14:29:43,209 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2026-05-11T14:29:43,213 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,214 copying build/lib/lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,216 copying build/lib/lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,218 copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,219 copying build/lib/lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,221 copying build/lib/lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,223 copying build/lib/lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,224 copying build/lib/lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,226 copying build/lib/lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,228 copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,230 copying build/lib/lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,231 copying build/lib/lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,233 copying build/lib/lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,235 copying build/lib/lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,236 copying build/lib/lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,238 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,240 copying build/lib/lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,242 copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,244 copying build/lib/lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,245 copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,247 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,249 copying build/lib/lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,251 copying build/lib/lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,252 copying build/lib/lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,254 copying build/lib/lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,256 copying build/lib/lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,258 copying build/lib/lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,259 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,261 copying build/lib/lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,263 copying build/lib/lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,265 copying build/lib/lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,266 copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,268 copying build/lib/lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,270 copying build/lib/lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,271 copying build/lib/lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,273 copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,275 copying build/lib/lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,276 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,278 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,280 copying build/lib/lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,282 copying build/lib/lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,284 copying build/lib/lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,285 copying build/lib/lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,287 copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,289 copying build/lib/lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,291 copying build/lib/lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,292 copying build/lib/lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,294 copying build/lib/lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,296 copying build/lib/lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,298 copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,300 copying build/lib/lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,301 copying build/lib/lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,303 copying build/lib/lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,305 copying build/lib/lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,307 copying build/lib/lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,309 copying build/lib/lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,310 copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,312 copying build/lib/lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,314 copying build/lib/lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,315 copying build/lib/lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,317 copying build/lib/lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,319 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,321 copying build/lib/lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,322 copying build/lib/lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,324 copying build/lib/lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,326 copying build/lib/lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,328 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,329 copying build/lib/lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,331 copying build/lib/lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,333 copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,335 copying build/lib/lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,336 copying build/lib/lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,338 copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,340 copying build/lib/lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,342 copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,344 copying build/lib/lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,346 copying build/lib/lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,348 copying build/lib/lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,350 copying build/lib/lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,352 copying build/lib/lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,354 copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,356 copying build/lib/lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,358 copying build/lib/lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,360 copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,362 copying build/lib/lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,364 copying build/lib/lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,366 copying build/lib/lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,368 copying build/lib/lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,369 copying build/lib/lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,371 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,373 copying build/lib/lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,375 copying build/lib/lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,377 copying build/lib/lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,379 copying build/lib/lm_eval/tasks/bigbench/generate_until/color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,381 copying build/lib/lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,383 copying build/lib/lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,384 copying build/lib/lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,386 copying build/lib/lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,388 copying build/lib/lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,390 copying build/lib/lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,391 copying build/lib/lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,393 copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,394 copying build/lib/lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,396 copying build/lib/lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,398 copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,399 copying build/lib/lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,401 copying build/lib/lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,403 copying build/lib/lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,405 copying build/lib/lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,406 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,408 copying build/lib/lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,410 copying build/lib/lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,412 copying build/lib/lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,413 copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,415 copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,417 copying build/lib/lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,419 copying build/lib/lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,421 copying build/lib/lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,422 copying build/lib/lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,424 copying build/lib/lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,426 copying build/lib/lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,428 copying build/lib/lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,429 copying build/lib/lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,431 copying build/lib/lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,433 copying build/lib/lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,434 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,436 copying build/lib/lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,438 copying build/lib/lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,439 copying build/lib/lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,441 copying build/lib/lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,443 copying build/lib/lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,444 copying build/lib/lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,446 copying build/lib/lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,448 copying build/lib/lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,450 copying build/lib/lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,451 copying build/lib/lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,453 copying build/lib/lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,455 copying build/lib/lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,456 copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,458 copying build/lib/lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,460 copying build/lib/lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,462 copying build/lib/lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,463 copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,465 copying build/lib/lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,467 copying build/lib/lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,468 copying build/lib/lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,470 copying build/lib/lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,472 copying build/lib/lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,473 copying build/lib/lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,475 copying build/lib/lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,477 copying build/lib/lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,478 copying build/lib/lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,480 copying build/lib/lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,481 copying build/lib/lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,483 copying build/lib/lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,485 copying build/lib/lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,487 copying build/lib/lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,488 copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,490 copying build/lib/lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,492 copying build/lib/lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,494 copying build/lib/lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,495 copying build/lib/lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,497 copying build/lib/lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,499 copying build/lib/lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,501 copying build/lib/lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,502 copying build/lib/lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,504 copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,506 copying build/lib/lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T14:29:43,507 copying build/lib/lm_eval/tasks/bigbench/generate_tasks.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T14:29:43,512 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,513 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,515 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,516 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,518 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,520 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,521 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,523 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,525 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,527 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,528 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,530 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,532 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,534 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,535 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,537 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,539 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,541 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,543 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,544 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,546 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,548 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,550 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,551 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,553 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,555 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,556 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,558 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,560 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,561 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,563 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,565 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,566 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,568 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,570 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,571 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,573 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,575 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,577 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,578 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,580 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,582 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,584 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,586 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,588 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,590 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,593 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,594 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,596 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,598 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,599 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,601 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,603 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,605 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,607 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,608 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,610 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,612 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,614 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,616 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,617 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,619 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,621 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,623 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,625 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,627 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,629 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,631 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,632 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,634 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,636 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,637 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,639 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,641 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,642 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,644 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,646 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,647 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,649 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,651 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,653 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,655 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,657 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,659 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,662 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,664 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,667 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,669 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,671 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,672 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,674 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,675 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,677 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,679 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,680 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,682 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,684 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,685 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,687 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,689 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,691 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,692 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,694 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,696 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,698 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,700 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,701 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,703 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,705 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,707 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,709 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,711 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,712 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,714 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,716 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,717 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,719 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,720 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,722 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,724 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T14:29:43,726 copying build/lib/lm_eval/tasks/bigbench/multiple_choice_template_a_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T14:29:43,728 copying build/lib/lm_eval/tasks/bigbench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T14:29:43,730 copying build/lib/lm_eval/tasks/bigbench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T14:29:43,732 copying build/lib/lm_eval/tasks/bigbench/multiple_choice_template_b_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T14:29:43,734 copying build/lib/lm_eval/tasks/bigbench/generate_until_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T14:29:43,736 copying build/lib/lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T14:29:43,738 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_piqa 2026-05-11T14:29:43,741 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,742 copying build/lib/lm_eval/tasks/global_piqa/completions/fin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,744 copying build/lib/lm_eval/tasks/global_piqa/completions/azj_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,747 copying build/lib/lm_eval/tasks/global_piqa/completions/spa_latn_mexi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,749 copying build/lib/lm_eval/tasks/global_piqa/completions/slk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,750 copying build/lib/lm_eval/tasks/global_piqa/completions/deu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,752 copying build/lib/lm_eval/tasks/global_piqa/completions/nob_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,754 copying build/lib/lm_eval/tasks/global_piqa/completions/swe_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,755 copying build/lib/lm_eval/tasks/global_piqa/completions/jpn_jpan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,757 copying build/lib/lm_eval/tasks/global_piqa/completions/lin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,759 copying build/lib/lm_eval/tasks/global_piqa/completions/glg_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,760 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_syri.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,762 copying build/lib/lm_eval/tasks/global_piqa/completions/ars_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,764 copying build/lib/lm_eval/tasks/global_piqa/completions/slv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,765 copying build/lib/lm_eval/tasks/global_piqa/completions/por_latn_port.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,767 copying build/lib/lm_eval/tasks/global_piqa/completions/_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,769 copying build/lib/lm_eval/tasks/global_piqa/completions/bsk_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,770 copying build/lib/lm_eval/tasks/global_piqa/completions/tur_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,772 copying build/lib/lm_eval/tasks/global_piqa/completions/zsm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,774 copying build/lib/lm_eval/tasks/global_piqa/completions/ekk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,776 copying build/lib/lm_eval/tasks/global_piqa/completions/tgl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,778 copying build/lib/lm_eval/tasks/global_piqa/completions/pes_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,780 copying build/lib/lm_eval/tasks/global_piqa/completions/tam_taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,781 copying build/lib/lm_eval/tasks/global_piqa/completions/hau_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,783 copying build/lib/lm_eval/tasks/global_piqa/completions/kan_knda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,785 copying build/lib/lm_eval/tasks/global_piqa/completions/pan_guru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,787 copying build/lib/lm_eval/tasks/global_piqa/completions/ben_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,789 copying build/lib/lm_eval/tasks/global_piqa/completions/swh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,790 copying build/lib/lm_eval/tasks/global_piqa/completions/sin_sinh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,792 copying build/lib/lm_eval/tasks/global_piqa/completions/mkd_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,794 copying build/lib/lm_eval/tasks/global_piqa/completions/bel_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,796 copying build/lib/lm_eval/tasks/global_piqa/completions/mni_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,798 copying build/lib/lm_eval/tasks/global_piqa/completions/cmn_hans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,800 copying build/lib/lm_eval/tasks/global_piqa/completions/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,802 copying build/lib/lm_eval/tasks/global_piqa/completions/mni_mtei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,803 copying build/lib/lm_eval/tasks/global_piqa/completions/hye_armn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,805 copying build/lib/lm_eval/tasks/global_piqa/completions/yor_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,807 copying build/lib/lm_eval/tasks/global_piqa/completions/fra_latn_fran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,808 copying build/lib/lm_eval/tasks/global_piqa/completions/ben_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,810 copying build/lib/lm_eval/tasks/global_piqa/completions/slv_latn_cerk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,812 copying build/lib/lm_eval/tasks/global_piqa/completions/als_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,813 copying build/lib/lm_eval/tasks/global_piqa/completions/arb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,815 copying build/lib/lm_eval/tasks/global_piqa/completions/hun_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,817 copying build/lib/lm_eval/tasks/global_piqa/completions/aeb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,819 copying build/lib/lm_eval/tasks/global_piqa/completions/ibo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,820 copying build/lib/lm_eval/tasks/global_piqa/completions/kat_geor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,822 copying build/lib/lm_eval/tasks/global_piqa/completions/idu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,824 copying build/lib/lm_eval/tasks/global_piqa/completions/guj_gujr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,826 copying build/lib/lm_eval/tasks/global_piqa/completions/cmn_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,827 copying build/lib/lm_eval/tasks/global_piqa/completions/jav_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,829 copying build/lib/lm_eval/tasks/global_piqa/completions/mal_mlym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,831 copying build/lib/lm_eval/tasks/global_piqa/completions/ces_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,833 copying build/lib/lm_eval/tasks/global_piqa/completions/tel_telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,834 copying build/lib/lm_eval/tasks/global_piqa/completions/fra_latn_cana.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,836 copying build/lib/lm_eval/tasks/global_piqa/completions/arq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,838 copying build/lib/lm_eval/tasks/global_piqa/completions/afb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,840 copying build/lib/lm_eval/tasks/global_piqa/completions/kor_hang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,841 copying build/lib/lm_eval/tasks/global_piqa/completions/ary_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,843 copying build/lib/lm_eval/tasks/global_piqa/completions/ckm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,844 copying build/lib/lm_eval/tasks/global_piqa/completions/haw_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,846 copying build/lib/lm_eval/tasks/global_piqa/completions/hrv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,848 copying build/lib/lm_eval/tasks/global_piqa/completions/bul_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,850 copying build/lib/lm_eval/tasks/global_piqa/completions/ekp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,852 copying build/lib/lm_eval/tasks/global_piqa/completions/iso_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,853 copying build/lib/lm_eval/tasks/global_piqa/completions/slk_latn_sari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,855 copying build/lib/lm_eval/tasks/global_piqa/completions/asm_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,860 copying build/lib/lm_eval/tasks/global_piqa/completions/lit_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,862 copying build/lib/lm_eval/tasks/global_piqa/completions/ukr_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,863 copying build/lib/lm_eval/tasks/global_piqa/completions/nag_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,865 copying build/lib/lm_eval/tasks/global_piqa/completions/yue_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,867 copying build/lib/lm_eval/tasks/global_piqa/completions/dhd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,869 copying build/lib/lm_eval/tasks/global_piqa/completions/kir_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,870 copying build/lib/lm_eval/tasks/global_piqa/completions/vie_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,872 copying build/lib/lm_eval/tasks/global_piqa/completions/amh_ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,874 copying build/lib/lm_eval/tasks/global_piqa/completions/eng_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,876 copying build/lib/lm_eval/tasks/global_piqa/completions/luo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,878 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_jord.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,879 copying build/lib/lm_eval/tasks/global_piqa/completions/bho_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,881 copying build/lib/lm_eval/tasks/global_piqa/completions/bam_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,883 copying build/lib/lm_eval/tasks/global_piqa/completions/pol_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,885 copying build/lib/lm_eval/tasks/global_piqa/completions/arz_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,887 copying build/lib/lm_eval/tasks/global_piqa/completions/srp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,888 copying build/lib/lm_eval/tasks/global_piqa/completions/urd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,890 copying build/lib/lm_eval/tasks/global_piqa/completions/cat_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,892 copying build/lib/lm_eval/tasks/global_piqa/completions/ckb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,893 copying build/lib/lm_eval/tasks/global_piqa/completions/ita_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,896 copying build/lib/lm_eval/tasks/global_piqa/completions/srp_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,898 copying build/lib/lm_eval/tasks/global_piqa/completions/pcm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,899 copying build/lib/lm_eval/tasks/global_piqa/completions/ind_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,901 copying build/lib/lm_eval/tasks/global_piqa/completions/npi_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,903 copying build/lib/lm_eval/tasks/global_piqa/completions/urd_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,904 copying build/lib/lm_eval/tasks/global_piqa/completions/heb_hebr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,906 copying build/lib/lm_eval/tasks/global_piqa/completions/rwr_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,908 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_leba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,910 copying build/lib/lm_eval/tasks/global_piqa/completions/_global_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,912 copying build/lib/lm_eval/tasks/global_piqa/completions/kaz_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,914 copying build/lib/lm_eval/tasks/global_piqa/completions/spa_latn_spai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,915 copying build/lib/lm_eval/tasks/global_piqa/completions/ron_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,917 copying build/lib/lm_eval/tasks/global_piqa/completions/mar_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,919 copying build/lib/lm_eval/tasks/global_piqa/completions/isl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,921 copying build/lib/lm_eval/tasks/global_piqa/completions/nld_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,923 copying build/lib/lm_eval/tasks/global_piqa/completions/snd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,924 copying build/lib/lm_eval/tasks/global_piqa/completions/ell_grek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,926 copying build/lib/lm_eval/tasks/global_piqa/completions/tha_thai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,928 copying build/lib/lm_eval/tasks/global_piqa/completions/urh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,931 copying build/lib/lm_eval/tasks/global_piqa/completions/hin_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,932 copying build/lib/lm_eval/tasks/global_piqa/completions/por_latn_braz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,934 copying build/lib/lm_eval/tasks/global_piqa/completions/uzn_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,936 copying build/lib/lm_eval/tasks/global_piqa/completions/acq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,938 copying build/lib/lm_eval/tasks/global_piqa/completions/kin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,940 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_pale.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,942 copying build/lib/lm_eval/tasks/global_piqa/completions/uig_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,944 copying build/lib/lm_eval/tasks/global_piqa/completions/spa_latn_peru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,946 copying build/lib/lm_eval/tasks/global_piqa/completions/bos_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,948 copying build/lib/lm_eval/tasks/global_piqa/completions/fao_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,950 copying build/lib/lm_eval/tasks/global_piqa/completions/acm_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,951 copying build/lib/lm_eval/tasks/global_piqa/completions/zul_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,953 copying build/lib/lm_eval/tasks/global_piqa/completions/rus_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,955 copying build/lib/lm_eval/tasks/global_piqa/completions/nno_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,957 copying build/lib/lm_eval/tasks/global_piqa/completions/snd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T14:29:43,959 copying build/lib/lm_eval/tasks/global_piqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa 2026-05-11T14:29:43,963 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,964 copying build/lib/lm_eval/tasks/global_piqa/prompted/fin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,965 copying build/lib/lm_eval/tasks/global_piqa/prompted/azj_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,967 copying build/lib/lm_eval/tasks/global_piqa/prompted/spa_latn_mexi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,969 copying build/lib/lm_eval/tasks/global_piqa/prompted/slk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,971 copying build/lib/lm_eval/tasks/global_piqa/prompted/deu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,973 copying build/lib/lm_eval/tasks/global_piqa/prompted/nob_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,974 copying build/lib/lm_eval/tasks/global_piqa/prompted/swe_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,976 copying build/lib/lm_eval/tasks/global_piqa/prompted/jpn_jpan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,978 copying build/lib/lm_eval/tasks/global_piqa/prompted/lin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,979 copying build/lib/lm_eval/tasks/global_piqa/prompted/glg_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,981 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_syri.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,983 copying build/lib/lm_eval/tasks/global_piqa/prompted/ars_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,984 copying build/lib/lm_eval/tasks/global_piqa/prompted/slv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,986 copying build/lib/lm_eval/tasks/global_piqa/prompted/por_latn_port.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,988 copying build/lib/lm_eval/tasks/global_piqa/prompted/_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,989 copying build/lib/lm_eval/tasks/global_piqa/prompted/bsk_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,991 copying build/lib/lm_eval/tasks/global_piqa/prompted/tur_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,993 copying build/lib/lm_eval/tasks/global_piqa/prompted/zsm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,995 copying build/lib/lm_eval/tasks/global_piqa/prompted/ekk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,996 copying build/lib/lm_eval/tasks/global_piqa/prompted/tgl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:43,998 copying build/lib/lm_eval/tasks/global_piqa/prompted/pes_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,000 copying build/lib/lm_eval/tasks/global_piqa/prompted/tam_taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,002 copying build/lib/lm_eval/tasks/global_piqa/prompted/hau_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,003 copying build/lib/lm_eval/tasks/global_piqa/prompted/kan_knda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,005 copying build/lib/lm_eval/tasks/global_piqa/prompted/pan_guru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,007 copying build/lib/lm_eval/tasks/global_piqa/prompted/ben_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,009 copying build/lib/lm_eval/tasks/global_piqa/prompted/swh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,010 copying build/lib/lm_eval/tasks/global_piqa/prompted/sin_sinh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,012 copying build/lib/lm_eval/tasks/global_piqa/prompted/mkd_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,014 copying build/lib/lm_eval/tasks/global_piqa/prompted/bel_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,016 copying build/lib/lm_eval/tasks/global_piqa/prompted/mni_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,017 copying build/lib/lm_eval/tasks/global_piqa/prompted/cmn_hans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,019 copying build/lib/lm_eval/tasks/global_piqa/prompted/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,021 copying build/lib/lm_eval/tasks/global_piqa/prompted/mni_mtei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,022 copying build/lib/lm_eval/tasks/global_piqa/prompted/hye_armn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,024 copying build/lib/lm_eval/tasks/global_piqa/prompted/yor_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,026 copying build/lib/lm_eval/tasks/global_piqa/prompted/fra_latn_fran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,027 copying build/lib/lm_eval/tasks/global_piqa/prompted/ben_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,029 copying build/lib/lm_eval/tasks/global_piqa/prompted/slv_latn_cerk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,031 copying build/lib/lm_eval/tasks/global_piqa/prompted/als_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,032 copying build/lib/lm_eval/tasks/global_piqa/prompted/arb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,034 copying build/lib/lm_eval/tasks/global_piqa/prompted/hun_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,036 copying build/lib/lm_eval/tasks/global_piqa/prompted/aeb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,037 copying build/lib/lm_eval/tasks/global_piqa/prompted/ibo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,039 copying build/lib/lm_eval/tasks/global_piqa/prompted/kat_geor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,041 copying build/lib/lm_eval/tasks/global_piqa/prompted/idu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,043 copying build/lib/lm_eval/tasks/global_piqa/prompted/guj_gujr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,044 copying build/lib/lm_eval/tasks/global_piqa/prompted/cmn_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,046 copying build/lib/lm_eval/tasks/global_piqa/prompted/jav_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,048 copying build/lib/lm_eval/tasks/global_piqa/prompted/mal_mlym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,050 copying build/lib/lm_eval/tasks/global_piqa/prompted/ces_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,052 copying build/lib/lm_eval/tasks/global_piqa/prompted/tel_telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,053 copying build/lib/lm_eval/tasks/global_piqa/prompted/fra_latn_cana.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,055 copying build/lib/lm_eval/tasks/global_piqa/prompted/arq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,057 copying build/lib/lm_eval/tasks/global_piqa/prompted/afb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,059 copying build/lib/lm_eval/tasks/global_piqa/prompted/kor_hang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,060 copying build/lib/lm_eval/tasks/global_piqa/prompted/ary_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,062 copying build/lib/lm_eval/tasks/global_piqa/prompted/ckm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,064 copying build/lib/lm_eval/tasks/global_piqa/prompted/haw_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,065 copying build/lib/lm_eval/tasks/global_piqa/prompted/hrv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,067 copying build/lib/lm_eval/tasks/global_piqa/prompted/bul_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,069 copying build/lib/lm_eval/tasks/global_piqa/prompted/ekp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,070 copying build/lib/lm_eval/tasks/global_piqa/prompted/iso_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,072 copying build/lib/lm_eval/tasks/global_piqa/prompted/slk_latn_sari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,074 copying build/lib/lm_eval/tasks/global_piqa/prompted/asm_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,075 copying build/lib/lm_eval/tasks/global_piqa/prompted/lit_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,077 copying build/lib/lm_eval/tasks/global_piqa/prompted/ukr_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,079 copying build/lib/lm_eval/tasks/global_piqa/prompted/nag_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,081 copying build/lib/lm_eval/tasks/global_piqa/prompted/yue_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,082 copying build/lib/lm_eval/tasks/global_piqa/prompted/dhd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,084 copying build/lib/lm_eval/tasks/global_piqa/prompted/kir_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,086 copying build/lib/lm_eval/tasks/global_piqa/prompted/vie_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,088 copying build/lib/lm_eval/tasks/global_piqa/prompted/amh_ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,089 copying build/lib/lm_eval/tasks/global_piqa/prompted/eng_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,091 copying build/lib/lm_eval/tasks/global_piqa/prompted/luo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,093 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_jord.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,095 copying build/lib/lm_eval/tasks/global_piqa/prompted/bho_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,096 copying build/lib/lm_eval/tasks/global_piqa/prompted/bam_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,098 copying build/lib/lm_eval/tasks/global_piqa/prompted/pol_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,100 copying build/lib/lm_eval/tasks/global_piqa/prompted/arz_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,102 copying build/lib/lm_eval/tasks/global_piqa/prompted/srp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,103 copying build/lib/lm_eval/tasks/global_piqa/prompted/urd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,105 copying build/lib/lm_eval/tasks/global_piqa/prompted/cat_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,107 copying build/lib/lm_eval/tasks/global_piqa/prompted/ckb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,109 copying build/lib/lm_eval/tasks/global_piqa/prompted/ita_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,111 copying build/lib/lm_eval/tasks/global_piqa/prompted/srp_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,114 copying build/lib/lm_eval/tasks/global_piqa/prompted/pcm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,116 copying build/lib/lm_eval/tasks/global_piqa/prompted/ind_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,117 copying build/lib/lm_eval/tasks/global_piqa/prompted/npi_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,119 copying build/lib/lm_eval/tasks/global_piqa/prompted/urd_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,121 copying build/lib/lm_eval/tasks/global_piqa/prompted/heb_hebr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,123 copying build/lib/lm_eval/tasks/global_piqa/prompted/rwr_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,124 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_leba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,126 copying build/lib/lm_eval/tasks/global_piqa/prompted/_global_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,128 copying build/lib/lm_eval/tasks/global_piqa/prompted/kaz_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,130 copying build/lib/lm_eval/tasks/global_piqa/prompted/spa_latn_spai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,132 copying build/lib/lm_eval/tasks/global_piqa/prompted/ron_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,134 copying build/lib/lm_eval/tasks/global_piqa/prompted/mar_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,135 copying build/lib/lm_eval/tasks/global_piqa/prompted/isl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,137 copying build/lib/lm_eval/tasks/global_piqa/prompted/nld_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,139 copying build/lib/lm_eval/tasks/global_piqa/prompted/snd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,140 copying build/lib/lm_eval/tasks/global_piqa/prompted/ell_grek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,142 copying build/lib/lm_eval/tasks/global_piqa/prompted/tha_thai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,144 copying build/lib/lm_eval/tasks/global_piqa/prompted/urh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,146 copying build/lib/lm_eval/tasks/global_piqa/prompted/hin_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,148 copying build/lib/lm_eval/tasks/global_piqa/prompted/por_latn_braz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,150 copying build/lib/lm_eval/tasks/global_piqa/prompted/uzn_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,151 copying build/lib/lm_eval/tasks/global_piqa/prompted/acq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,153 copying build/lib/lm_eval/tasks/global_piqa/prompted/kin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,155 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_pale.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,156 copying build/lib/lm_eval/tasks/global_piqa/prompted/uig_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,158 copying build/lib/lm_eval/tasks/global_piqa/prompted/spa_latn_peru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,160 copying build/lib/lm_eval/tasks/global_piqa/prompted/bos_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,161 copying build/lib/lm_eval/tasks/global_piqa/prompted/fao_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,163 copying build/lib/lm_eval/tasks/global_piqa/prompted/acm_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,165 copying build/lib/lm_eval/tasks/global_piqa/prompted/zul_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,166 copying build/lib/lm_eval/tasks/global_piqa/prompted/rus_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,168 copying build/lib/lm_eval/tasks/global_piqa/prompted/nno_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,170 copying build/lib/lm_eval/tasks/global_piqa/prompted/snd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T14:29:44,172 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh 2026-05-11T14:29:44,173 copying build/lib/lm_eval/tasks/bbh/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh 2026-05-11T14:29:44,176 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,177 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,179 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,181 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,183 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_bbh_cot_fewshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,185 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,187 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,189 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,190 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,192 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,194 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,196 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,198 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,200 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,201 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,203 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,205 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,207 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_bbh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,209 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,210 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,212 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,214 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,216 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,218 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,219 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,221 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,224 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,226 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,227 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,229 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,231 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T14:29:44,234 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,235 copying build/lib/lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,237 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,239 copying build/lib/lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,241 copying build/lib/lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,243 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,245 copying build/lib/lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,246 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,248 copying build/lib/lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,250 copying build/lib/lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,251 copying build/lib/lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,253 copying build/lib/lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,255 copying build/lib/lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,257 copying build/lib/lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,258 copying build/lib/lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,260 copying build/lib/lm_eval/tasks/bbh/fewshot/_bbh_fewshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,262 copying build/lib/lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,263 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,265 copying build/lib/lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,267 copying build/lib/lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,269 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,271 copying build/lib/lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,272 copying build/lib/lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,274 copying build/lib/lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,276 copying build/lib/lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,278 copying build/lib/lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,279 copying build/lib/lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,281 copying build/lib/lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,283 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,285 copying build/lib/lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T14:29:44,288 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,289 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,291 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,292 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,294 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,296 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,297 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,299 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,301 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,302 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,304 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/_bbh_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,306 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,307 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,309 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,311 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,313 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,315 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,316 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,318 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,320 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,322 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,323 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,325 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,327 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,329 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,331 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,333 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,334 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,336 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,338 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,340 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T14:29:44,341 copying build/lib/lm_eval/tasks/bbh/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh 2026-05-11T14:29:44,344 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,345 copying build/lib/lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,347 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,349 copying build/lib/lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,350 copying build/lib/lm_eval/tasks/bbh/zeroshot/_bbh_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,352 copying build/lib/lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,354 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,355 copying build/lib/lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,357 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,359 copying build/lib/lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,361 copying build/lib/lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,363 copying build/lib/lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,364 copying build/lib/lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,366 copying build/lib/lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,368 copying build/lib/lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,370 copying build/lib/lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,372 copying build/lib/lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,374 copying build/lib/lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,375 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,377 copying build/lib/lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,379 copying build/lib/lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,380 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,382 copying build/lib/lm_eval/tasks/bbh/zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,384 copying build/lib/lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,386 copying build/lib/lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,388 copying build/lib/lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,389 copying build/lib/lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,391 copying build/lib/lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,392 copying build/lib/lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,394 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,396 copying build/lib/lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T14:29:44,398 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi 2026-05-11T14:29:44,399 copying build/lib/lm_eval/tasks/babi/babi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babi 2026-05-11T14:29:44,401 copying build/lib/lm_eval/tasks/babi/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babi 2026-05-11T14:29:44,403 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop 2026-05-11T14:29:44,404 copying build/lib/lm_eval/tasks/drop/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/drop 2026-05-11T14:29:44,406 copying build/lib/lm_eval/tasks/drop/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/drop 2026-05-11T14:29:44,408 copying build/lib/lm_eval/tasks/drop/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/drop 2026-05-11T14:29:44,411 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen 2026-05-11T14:29:44,412 copying build/lib/lm_eval/tasks/toxigen/toxigen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toxigen 2026-05-11T14:29:44,414 copying build/lib/lm_eval/tasks/toxigen/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toxigen 2026-05-11T14:29:44,415 copying build/lib/lm_eval/tasks/toxigen/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toxigen 2026-05-11T14:29:44,418 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa 2026-05-11T14:29:44,419 copying build/lib/lm_eval/tasks/pubmedqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pubmedqa 2026-05-11T14:29:44,421 copying build/lib/lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pubmedqa 2026-05-11T14:29:44,422 copying build/lib/lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pubmedqa 2026-05-11T14:29:44,425 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli 2026-05-11T14:29:44,425 copying build/lib/lm_eval/tasks/anli/anli_r3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T14:29:44,427 copying build/lib/lm_eval/tasks/anli/anli_r2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T14:29:44,429 copying build/lib/lm_eval/tasks/anli/anli_r1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T14:29:44,431 copying build/lib/lm_eval/tasks/anli/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T14:29:44,433 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/infinitebench 2026-05-11T14:29:44,434 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_math_find.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,436 copying build/lib/lm_eval/tasks/infinitebench/_infinitebench_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,438 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_code_debug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,440 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_longbook_qa_chn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,441 copying build/lib/lm_eval/tasks/infinitebench/infinitebench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,443 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_longbook_choice_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,445 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_number_string.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,446 copying build/lib/lm_eval/tasks/infinitebench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,448 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_passkey.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,450 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_code_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,452 copying build/lib/lm_eval/tasks/infinitebench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,454 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_longbook_qa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,456 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_longdialogue_qa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,457 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_longbook_sum_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,459 copying build/lib/lm_eval/tasks/infinitebench/infinitebench_kv_retrieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/infinitebench 2026-05-11T14:29:44,462 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2026-05-11T14:29:44,463 copying build/lib/lm_eval/tasks/agieval/agieval_cn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,465 copying build/lib/lm_eval/tasks/agieval/lsat-rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,467 copying build/lib/lm_eval/tasks/agieval/math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,469 copying build/lib/lm_eval/tasks/agieval/aqua-rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,471 copying build/lib/lm_eval/tasks/agieval/jec-qa-kd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,472 copying build/lib/lm_eval/tasks/agieval/lsat-ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,474 copying build/lib/lm_eval/tasks/agieval/agieval_nous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,476 copying build/lib/lm_eval/tasks/agieval/agieval_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,477 copying build/lib/lm_eval/tasks/agieval/logiqa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,479 copying build/lib/lm_eval/tasks/agieval/jec-qa-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,481 copying build/lib/lm_eval/tasks/agieval/gaokao-chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,482 copying build/lib/lm_eval/tasks/agieval/gaokao-english.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,484 copying build/lib/lm_eval/tasks/agieval/sat-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,485 copying build/lib/lm_eval/tasks/agieval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,488 copying build/lib/lm_eval/tasks/agieval/lsat-lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,489 copying build/lib/lm_eval/tasks/agieval/gaokao-chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,491 copying build/lib/lm_eval/tasks/agieval/gaokao-mathqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,493 copying build/lib/lm_eval/tasks/agieval/gaokao-biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,494 copying build/lib/lm_eval/tasks/agieval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,496 copying build/lib/lm_eval/tasks/agieval/gaokao-history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,498 copying build/lib/lm_eval/tasks/agieval/gaokao-mathcloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,500 copying build/lib/lm_eval/tasks/agieval/gaokao-physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,502 copying build/lib/lm_eval/tasks/agieval/logiqa-zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,503 copying build/lib/lm_eval/tasks/agieval/sat-en-without-passage.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,505 copying build/lib/lm_eval/tasks/agieval/agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,507 copying build/lib/lm_eval/tasks/agieval/sat-math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,509 copying build/lib/lm_eval/tasks/agieval/gaokao-geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T14:29:44,511 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost 2026-05-11T14:29:44,512 copying build/lib/lm_eval/tasks/prost/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/prost 2026-05-11T14:29:44,514 copying build/lib/lm_eval/tasks/prost/corypaik_prost.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/prost 2026-05-11T14:29:44,517 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2026-05-11T14:29:44,518 copying build/lib/lm_eval/tasks/unitxt/argument_topic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,520 copying build/lib/lm_eval/tasks/unitxt/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,523 copying build/lib/lm_eval/tasks/unitxt/ag_news.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,525 copying build/lib/lm_eval/tasks/unitxt/unitxt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,527 copying build/lib/lm_eval/tasks/unitxt/unfair_tos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,529 copying build/lib/lm_eval/tasks/unitxt/medical_abstracts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,531 copying build/lib/lm_eval/tasks/unitxt/unitxt_multimodal -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,532 copying build/lib/lm_eval/tasks/unitxt/dbpedia_14.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,534 copying build/lib/lm_eval/tasks/unitxt/ledgar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,536 copying build/lib/lm_eval/tasks/unitxt/financial_tweets.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,538 copying build/lib/lm_eval/tasks/unitxt/cnn_dailymail.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,540 copying build/lib/lm_eval/tasks/unitxt/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,542 copying build/lib/lm_eval/tasks/unitxt/xsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,543 copying build/lib/lm_eval/tasks/unitxt/yahoo_answers_topics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,545 copying build/lib/lm_eval/tasks/unitxt/law_stack_exchange.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,547 copying build/lib/lm_eval/tasks/unitxt/banking77.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,549 copying build/lib/lm_eval/tasks/unitxt/20_newsgroups.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,551 copying build/lib/lm_eval/tasks/unitxt/claim_stance_topic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,552 copying build/lib/lm_eval/tasks/unitxt/atis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,554 copying build/lib/lm_eval/tasks/unitxt/stsb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,556 copying build/lib/lm_eval/tasks/unitxt/ethos_binary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,557 copying build/lib/lm_eval/tasks/unitxt/coedit_gec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,559 copying build/lib/lm_eval/tasks/unitxt/doc_vqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T14:29:44,561 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals 2026-05-11T14:29:44,563 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T14:29:44,564 copying build/lib/lm_eval/tasks/model_written_evals/winogenerated/winogenerated -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T14:29:44,567 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,568 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,570 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,572 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,573 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,575 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,577 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,578 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,580 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,582 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,584 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,585 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,587 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,589 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,590 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,592 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,594 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,595 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,597 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,599 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,601 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,602 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,604 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,606 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,607 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,609 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,611 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,612 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,614 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,616 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,617 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,619 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,621 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,622 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,624 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,625 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,627 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,629 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,631 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,632 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,634 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,636 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,637 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,639 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,641 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,643 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,645 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,646 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,648 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,650 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,651 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,653 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T14:29:44,655 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:44,656 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:44,658 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:44,660 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T14:29:44,665 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,666 copying build/lib/lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,668 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,670 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,672 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,674 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,675 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,677 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,679 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,681 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,682 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,684 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,686 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,688 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,689 copying build/lib/lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,691 copying build/lib/lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,693 copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,694 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,696 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,698 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,699 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,701 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,703 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,705 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,706 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,708 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,710 copying build/lib/lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,712 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,713 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,715 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,717 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,719 copying build/lib/lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,720 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,722 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,724 copying build/lib/lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,725 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,727 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,729 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,730 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,732 copying build/lib/lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,734 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,735 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,737 copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,739 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,741 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,742 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,744 copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,746 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,747 copying build/lib/lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,749 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,751 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,752 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,754 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,756 copying build/lib/lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,758 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,759 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,761 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,762 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,764 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,766 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,767 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,769 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,771 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,772 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,774 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,776 copying build/lib/lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,777 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,779 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,781 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,782 copying build/lib/lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,784 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,786 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,788 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,789 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,791 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,793 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,794 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,796 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,798 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,799 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,801 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,803 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,804 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,806 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,808 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,809 copying build/lib/lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,811 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,812 copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,814 copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,816 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,818 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,820 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,821 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,823 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,824 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,826 copying build/lib/lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,828 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,829 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,831 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,832 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,834 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,836 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,838 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,839 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,841 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,843 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,844 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,846 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,848 copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,849 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,851 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,853 copying build/lib/lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,855 copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,857 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,858 copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,860 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,862 copying build/lib/lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,863 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,865 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,867 copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,868 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,870 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,871 copying build/lib/lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,873 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,875 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,877 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,878 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,880 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,883 copying build/lib/lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,885 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,887 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,889 copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,891 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,893 copying build/lib/lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,894 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,896 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,899 copying build/lib/lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,900 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T14:29:44,903 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/metabench 2026-05-11T14:29:44,904 copying build/lib/lm_eval/tasks/metabench/metabench_arc_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,906 copying build/lib/lm_eval/tasks/metabench/metabench_arc_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,907 copying build/lib/lm_eval/tasks/metabench/process_docs_permute.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,910 copying build/lib/lm_eval/tasks/metabench/metabench_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,911 copying build/lib/lm_eval/tasks/metabench/metabench_gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,913 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,915 copying build/lib/lm_eval/tasks/metabench/metabench_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,917 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,919 copying build/lib/lm_eval/tasks/metabench/metabench_arc_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,920 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,922 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,924 copying build/lib/lm_eval/tasks/metabench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,926 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,928 copying build/lib/lm_eval/tasks/metabench/metabench_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,930 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,932 copying build/lib/lm_eval/tasks/metabench/process_docs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,934 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,936 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,938 copying build/lib/lm_eval/tasks/metabench/metabench_gsm8k_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,940 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,941 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,943 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,944 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,946 copying build/lib/lm_eval/tasks/metabench/metabench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,948 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,949 copying build/lib/lm_eval/tasks/metabench/metabench_arc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,951 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,953 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,954 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T14:29:44,958 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2026-05-11T14:29:44,959 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_agronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,960 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,962 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,964 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,966 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,968 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,969 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,971 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,973 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,976 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,979 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,980 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,982 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,984 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,985 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_legal_and_moral_basis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,987 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,989 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_ethnology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,990 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_ancient_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,992 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_engineering_hydrology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,994 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,995 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_logical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,997 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:44,998 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_security_study.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,000 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,001 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,003 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_food_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,005 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,006 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,008 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,010 copying build/lib/lm_eval/tasks/cmmlu/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,011 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,013 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,015 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_commonsense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,016 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,018 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,020 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_arts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,022 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,023 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,025 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,027 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_information_and_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,029 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,031 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,032 copying build/lib/lm_eval/tasks/cmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,034 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,036 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_journalism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,038 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,039 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,041 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_food_culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,043 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_actuarial_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,045 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,046 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,048 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_civil_service_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,050 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,052 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,053 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,057 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_medical_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,058 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,060 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,062 copying build/lib/lm_eval/tasks/cmmlu/_cmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,063 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_driving_rule.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,065 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,067 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,069 copying build/lib/lm_eval/tasks/cmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,071 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,073 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,075 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,077 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,080 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,082 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_construction_project_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,084 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,086 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T14:29:45,089 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa 2026-05-11T14:29:45,090 copying build/lib/lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medmcqa 2026-05-11T14:29:45,092 copying build/lib/lm_eval/tasks/medmcqa/medmcqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medmcqa 2026-05-11T14:29:45,095 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2026-05-11T14:29:45,096 copying build/lib/lm_eval/tasks/paws-x/paws_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,098 copying build/lib/lm_eval/tasks/paws-x/pawsx_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,100 copying build/lib/lm_eval/tasks/paws-x/_pawsx.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,102 copying build/lib/lm_eval/tasks/paws-x/paws_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,104 copying build/lib/lm_eval/tasks/paws-x/paws_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,106 copying build/lib/lm_eval/tasks/paws-x/paws_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,107 copying build/lib/lm_eval/tasks/paws-x/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,109 copying build/lib/lm_eval/tasks/paws-x/paws_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,111 copying build/lib/lm_eval/tasks/paws-x/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,113 copying build/lib/lm_eval/tasks/paws-x/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,115 copying build/lib/lm_eval/tasks/paws-x/paws_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,117 copying build/lib/lm_eval/tasks/paws-x/paws_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T14:29:45,119 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include 2026-05-11T14:29:45,121 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en 2026-05-11T14:29:45,123 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,124 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,126 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/_indonesian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,128 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,131 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,135 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,137 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,138 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/_include_base_44_indonesian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,140 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T14:29:45,142 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:45,144 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:45,145 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/_include_base_44_tagalog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:45,147 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/_tagalog_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:45,149 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:45,151 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T14:29:45,153 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,154 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/_north macedonian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,156 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,158 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,160 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,162 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,164 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/_include_base_44_north macedonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,165 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T14:29:45,168 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,169 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,171 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,173 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/_malayalam_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,174 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,176 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,178 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,180 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,182 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,184 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/_include_base_44_malayalam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T14:29:45,187 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:45,188 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:45,189 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:45,191 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:45,193 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/_include_base_44_belarusian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:45,195 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:45,197 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/_belarusian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T14:29:45,199 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:45,200 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/_kazakh_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:45,202 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:45,204 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:45,206 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/_include_base_44_kazakh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T14:29:45,209 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,210 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,213 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/_include_base_44_persian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,215 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,217 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,220 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/_persian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,222 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,224 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,226 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T14:29:45,229 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,230 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,232 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/_arabic_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,234 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,237 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,239 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,241 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/_include_base_44_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,244 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,247 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,249 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T14:29:45,252 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:45,253 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:45,255 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/_include_base_44_ukrainian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:45,257 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:45,259 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:45,262 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:45,264 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/_ukrainian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T14:29:45,267 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,268 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,270 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/_telugu_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,273 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,275 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,277 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,279 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,281 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/_include_base_44_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T14:29:45,284 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:45,285 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/_include_base_44_urdu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:45,288 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/_urdu_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:45,290 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:45,293 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:45,295 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:45,297 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T14:29:45,300 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,301 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,304 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/_include_base_44_lithuanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,306 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/_lithuanian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,308 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,309 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,311 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,313 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,315 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T14:29:45,318 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:45,319 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/_include_base_44_vietnamese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:45,321 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/_vietnamese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:45,322 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:45,324 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:45,326 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:45,328 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T14:29:45,330 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,331 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,333 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,334 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,336 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,338 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,340 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/_albanian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,341 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,343 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/_include_base_44_albanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T14:29:45,345 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,346 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/_finnish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,349 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,350 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,352 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,354 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,356 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,358 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,360 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/_include_base_44_finnish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T14:29:45,362 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,363 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/_spanish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,365 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,367 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,368 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,370 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,372 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/_include_base_44_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,374 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T14:29:45,376 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:45,377 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:45,379 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/_nepali_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:45,381 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/_include_base_44_nepali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:45,383 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:45,384 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T14:29:45,387 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:45,388 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:45,389 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/_include_base_44_polish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:45,391 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:45,393 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:45,395 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:45,397 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/_polish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T14:29:45,399 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,400 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/_azerbaijani_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,401 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,403 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,405 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,407 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,408 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,410 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,412 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,414 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T14:29:45,418 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:45,420 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:45,424 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:45,425 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/_include_base_44_serbian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:45,427 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/_serbian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:45,429 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:45,431 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T14:29:45,434 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,435 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,437 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/_include_base_44_portuguese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,438 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,440 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,442 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,443 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,445 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,447 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/_portuguese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,449 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T14:29:45,451 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,452 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,454 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,456 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,457 copying build/lib/lm_eval/tasks/include/few_shot_en/French/_french_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,459 copying build/lib/lm_eval/tasks/include/few_shot_en/French/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,461 copying build/lib/lm_eval/tasks/include/few_shot_en/French/_include_base_44_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,463 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,465 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T14:29:45,467 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,468 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,470 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,472 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/_uzbek_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,474 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,475 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,477 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/_include_base_44_uzbek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,479 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T14:29:45,481 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:45,482 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:45,483 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:45,485 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/_malay_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:45,487 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:45,489 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:45,490 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/_include_base_44_malay.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T14:29:45,493 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,494 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,496 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,497 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,499 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,501 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/_include_base_44_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,502 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,504 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,506 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/_italian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,508 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T14:29:45,510 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,511 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/_estonian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,513 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,515 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,517 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,518 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,520 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,522 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/_include_base_44_estonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,523 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T14:29:45,526 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,527 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,528 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/_bengali_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,530 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/_include_base_44_bengali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,532 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,534 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,535 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,537 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T14:29:45,539 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:45,540 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:45,542 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/_georgian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:45,544 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/_include_base_44_georgian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:45,546 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T14:29:45,548 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,549 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,551 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,553 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,554 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,556 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/_russian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,558 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/_include_base_44_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,559 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,561 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,563 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,564 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,566 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T14:29:45,569 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:45,570 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/_include_base_44_korean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:45,571 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/_korean_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:45,573 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:45,575 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:45,577 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T14:29:45,579 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,580 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,584 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,586 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,588 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,590 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,591 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/_chinese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,593 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,595 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,596 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/_include_base_44_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,598 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,600 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T14:29:45,602 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:45,603 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:45,605 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:45,607 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/_croatian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:45,609 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/_include_base_44_croatian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:45,611 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:45,613 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T14:29:45,615 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,616 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,618 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,620 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,621 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,623 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,625 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,627 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/_dutch_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,628 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/_include_base_44_dutch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T14:29:45,631 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,632 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,634 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,635 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,637 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,639 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,641 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,643 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/_include_base_44_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,645 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,646 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,648 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/_hindi_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,651 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T14:29:45,653 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:45,655 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/_tamil_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:45,657 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:45,659 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:45,661 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:45,663 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/_include_base_44_tamil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T14:29:45,665 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:45,666 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:45,668 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:45,670 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/_include_base_44_hungarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:45,672 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:45,673 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:45,675 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/_hungarian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T14:29:45,677 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,678 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/_include_base_44_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,680 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,682 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,683 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/_turkish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,685 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,687 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,689 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T14:29:45,691 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,692 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/_greek_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,694 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,695 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,697 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,699 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/_include_base_44_greek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,701 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,702 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,704 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,706 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,707 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T14:29:45,709 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,710 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/_include_base_44_armenian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,712 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,714 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,716 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,717 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,719 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,721 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/_armenian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T14:29:45,723 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:45,724 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/_hebrew_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:45,726 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:45,728 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:45,729 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/_include_base_44_hebrew.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:45,731 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T14:29:45,733 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:45,734 copying build/lib/lm_eval/tasks/include/few_shot_en/German/include_base_44_german_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:45,736 copying build/lib/lm_eval/tasks/include/few_shot_en/German/include_base_44_german_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:45,738 copying build/lib/lm_eval/tasks/include/few_shot_en/German/include_base_44_german_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:45,740 copying build/lib/lm_eval/tasks/include/few_shot_en/German/_german_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:45,741 copying build/lib/lm_eval/tasks/include/few_shot_en/German/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:45,743 copying build/lib/lm_eval/tasks/include/few_shot_en/German/_include_base_44_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T14:29:45,745 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:45,746 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:45,748 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/_japanese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:45,750 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:45,751 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:45,753 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:45,755 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/_include_base_44_japanese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T14:29:45,757 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:45,758 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/include_base_44_basque_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:45,760 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/_basque_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:45,761 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:45,763 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/_include_base_44_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T14:29:45,766 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:45,767 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/_include_base_44_bulgarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:45,769 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:45,770 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:45,772 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/_bulgarian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:45,774 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:45,775 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T14:29:45,777 copying build/lib/lm_eval/tasks/include/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include 2026-05-11T14:29:45,780 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default 2026-05-11T14:29:45,782 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,783 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,785 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,786 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,788 copying build/lib/lm_eval/tasks/include/default/Indonesian/_indonesian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,790 copying build/lib/lm_eval/tasks/include/default/Indonesian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,791 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,793 copying build/lib/lm_eval/tasks/include/default/Indonesian/_include_base_44_indonesian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,795 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T14:29:45,797 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:45,798 copying build/lib/lm_eval/tasks/include/default/Tagalog/_tagalog_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:45,800 copying build/lib/lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:45,802 copying build/lib/lm_eval/tasks/include/default/Tagalog/_include_base_44_tagalog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:45,803 copying build/lib/lm_eval/tasks/include/default/Tagalog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:45,805 copying build/lib/lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T14:29:45,807 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,808 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,810 copying build/lib/lm_eval/tasks/include/default/North Macedonian/_north macedonian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,812 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,814 copying build/lib/lm_eval/tasks/include/default/North Macedonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,815 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,817 copying build/lib/lm_eval/tasks/include/default/North Macedonian/_include_base_44_north macedonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,819 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T14:29:45,821 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,822 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,824 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,826 copying build/lib/lm_eval/tasks/include/default/Malayalam/_malayalam_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,827 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,830 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,831 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,833 copying build/lib/lm_eval/tasks/include/default/Malayalam/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,835 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,837 copying build/lib/lm_eval/tasks/include/default/Malayalam/_include_base_44_malayalam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T14:29:45,839 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:45,840 copying build/lib/lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:45,842 copying build/lib/lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:45,843 copying build/lib/lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:45,845 copying build/lib/lm_eval/tasks/include/default/Belarusian/_include_base_44_belarusian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:45,847 copying build/lib/lm_eval/tasks/include/default/Belarusian/_belarusian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:45,849 copying build/lib/lm_eval/tasks/include/default/Belarusian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T14:29:45,851 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:45,852 copying build/lib/lm_eval/tasks/include/default/Kazakh/_kazakh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:45,854 copying build/lib/lm_eval/tasks/include/default/Kazakh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:45,855 copying build/lib/lm_eval/tasks/include/default/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:45,857 copying build/lib/lm_eval/tasks/include/default/Kazakh/_include_base_44_kazakh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T14:29:45,859 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,860 copying build/lib/lm_eval/tasks/include/default/Persian/_persian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,862 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,864 copying build/lib/lm_eval/tasks/include/default/Persian/_include_base_44_persian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,865 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,867 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,869 copying build/lib/lm_eval/tasks/include/default/Persian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,870 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,872 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T14:29:45,874 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,875 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,877 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,879 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,881 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,882 copying build/lib/lm_eval/tasks/include/default/Arabic/_arabic_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,884 copying build/lib/lm_eval/tasks/include/default/Arabic/_include_base_44_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,886 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,888 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,889 copying build/lib/lm_eval/tasks/include/default/Arabic/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T14:29:45,892 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:45,892 copying build/lib/lm_eval/tasks/include/default/Ukrainian/_ukrainian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:45,894 copying build/lib/lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:45,896 copying build/lib/lm_eval/tasks/include/default/Ukrainian/_include_base_44_ukrainian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:45,898 copying build/lib/lm_eval/tasks/include/default/Ukrainian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:45,899 copying build/lib/lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:45,901 copying build/lib/lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T14:29:45,903 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,904 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,906 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,907 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,909 copying build/lib/lm_eval/tasks/include/default/Telugu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,912 copying build/lib/lm_eval/tasks/include/default/Telugu/_telugu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,913 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,915 copying build/lib/lm_eval/tasks/include/default/Telugu/_include_base_44_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T14:29:45,917 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:45,918 copying build/lib/lm_eval/tasks/include/default/Urdu/_include_base_44_urdu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:45,920 copying build/lib/lm_eval/tasks/include/default/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:45,922 copying build/lib/lm_eval/tasks/include/default/Urdu/_urdu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:45,923 copying build/lib/lm_eval/tasks/include/default/Urdu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:45,925 copying build/lib/lm_eval/tasks/include/default/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:45,927 copying build/lib/lm_eval/tasks/include/default/Urdu/include_base_44_urdu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T14:29:45,929 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,930 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,932 copying build/lib/lm_eval/tasks/include/default/Lithuanian/_include_base_44_lithuanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,934 copying build/lib/lm_eval/tasks/include/default/Lithuanian/_lithuanian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,935 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,937 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,938 copying build/lib/lm_eval/tasks/include/default/Lithuanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,940 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,942 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T14:29:45,944 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:45,945 copying build/lib/lm_eval/tasks/include/default/Vietnamese/_include_base_44_vietnamese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:45,947 copying build/lib/lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:45,948 copying build/lib/lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:45,950 copying build/lib/lm_eval/tasks/include/default/Vietnamese/_vietnamese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:45,952 copying build/lib/lm_eval/tasks/include/default/Vietnamese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:45,953 copying build/lib/lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T14:29:45,956 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,957 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,959 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,960 copying build/lib/lm_eval/tasks/include/default/Albanian/_albanian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,962 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,964 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,965 copying build/lib/lm_eval/tasks/include/default/Albanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,967 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,969 copying build/lib/lm_eval/tasks/include/default/Albanian/_include_base_44_albanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T14:29:45,971 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,972 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,974 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,976 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,977 copying build/lib/lm_eval/tasks/include/default/Finnish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,979 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,981 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,982 copying build/lib/lm_eval/tasks/include/default/Finnish/_include_base_44_finnish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,984 copying build/lib/lm_eval/tasks/include/default/Finnish/_finnish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T14:29:45,986 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:45,987 copying build/lib/lm_eval/tasks/include/default/Spanish/_spanish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:45,989 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:45,991 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:45,992 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:45,994 copying build/lib/lm_eval/tasks/include/default/Spanish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:45,996 copying build/lib/lm_eval/tasks/include/default/Spanish/_include_base_44_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:45,997 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T14:29:46,000 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:46,001 copying build/lib/lm_eval/tasks/include/default/Nepali/include_base_44_nepali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:46,003 copying build/lib/lm_eval/tasks/include/default/Nepali/_include_base_44_nepali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:46,004 copying build/lib/lm_eval/tasks/include/default/Nepali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:46,006 copying build/lib/lm_eval/tasks/include/default/Nepali/include_base_44_nepali_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:46,009 copying build/lib/lm_eval/tasks/include/default/Nepali/_nepali_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T14:29:46,011 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Polish 2026-05-11T14:29:46,012 copying build/lib/lm_eval/tasks/include/default/Polish/include_base_44_polish_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T14:29:46,014 copying build/lib/lm_eval/tasks/include/default/Polish/_include_base_44_polish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T14:29:46,015 copying build/lib/lm_eval/tasks/include/default/Polish/include_base_44_polish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T14:29:46,017 copying build/lib/lm_eval/tasks/include/default/Polish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T14:29:46,019 copying build/lib/lm_eval/tasks/include/default/Polish/include_base_44_polish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T14:29:46,020 copying build/lib/lm_eval/tasks/include/default/Polish/_polish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T14:29:46,023 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,023 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,025 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,027 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,029 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,030 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,032 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,034 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,036 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,037 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/_azerbaijani_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T14:29:46,040 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:46,041 copying build/lib/lm_eval/tasks/include/default/Serbian/include_base_44_serbian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:46,043 copying build/lib/lm_eval/tasks/include/default/Serbian/include_base_44_serbian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:46,044 copying build/lib/lm_eval/tasks/include/default/Serbian/_include_base_44_serbian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:46,046 copying build/lib/lm_eval/tasks/include/default/Serbian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:46,048 copying build/lib/lm_eval/tasks/include/default/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:46,050 copying build/lib/lm_eval/tasks/include/default/Serbian/_serbian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T14:29:46,052 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,053 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,055 copying build/lib/lm_eval/tasks/include/default/Portuguese/_include_base_44_portuguese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,057 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,058 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,060 copying build/lib/lm_eval/tasks/include/default/Portuguese/_portuguese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,062 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,063 copying build/lib/lm_eval/tasks/include/default/Portuguese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,065 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,067 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T14:29:46,069 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/French 2026-05-11T14:29:46,070 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,072 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,074 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,075 copying build/lib/lm_eval/tasks/include/default/French/_french_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,077 copying build/lib/lm_eval/tasks/include/default/French/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,079 copying build/lib/lm_eval/tasks/include/default/French/_include_base_44_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,081 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,082 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T14:29:46,085 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,086 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,088 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,089 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,091 copying build/lib/lm_eval/tasks/include/default/Uzbek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,092 copying build/lib/lm_eval/tasks/include/default/Uzbek/_include_base_44_uzbek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,094 copying build/lib/lm_eval/tasks/include/default/Uzbek/_uzbek_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,096 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T14:29:46,098 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Malay 2026-05-11T14:29:46,099 copying build/lib/lm_eval/tasks/include/default/Malay/include_base_44_malay_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T14:29:46,101 copying build/lib/lm_eval/tasks/include/default/Malay/_malay_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T14:29:46,102 copying build/lib/lm_eval/tasks/include/default/Malay/include_base_44_malay_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T14:29:46,104 copying build/lib/lm_eval/tasks/include/default/Malay/include_base_44_malay_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T14:29:46,106 copying build/lib/lm_eval/tasks/include/default/Malay/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T14:29:46,107 copying build/lib/lm_eval/tasks/include/default/Malay/_include_base_44_malay.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T14:29:46,110 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,111 copying build/lib/lm_eval/tasks/include/default/Italian/_italian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,113 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,114 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,116 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,118 copying build/lib/lm_eval/tasks/include/default/Italian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,119 copying build/lib/lm_eval/tasks/include/default/Italian/_include_base_44_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,121 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,123 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,125 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T14:29:46,127 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,128 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,130 copying build/lib/lm_eval/tasks/include/default/Estonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,132 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,133 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,135 copying build/lib/lm_eval/tasks/include/default/Estonian/_estonian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,137 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,138 copying build/lib/lm_eval/tasks/include/default/Estonian/_include_base_44_estonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,140 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T14:29:46,142 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,143 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,145 copying build/lib/lm_eval/tasks/include/default/Bengali/_bengali_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,147 copying build/lib/lm_eval/tasks/include/default/Bengali/_include_base_44_bengali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,149 copying build/lib/lm_eval/tasks/include/default/Bengali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,150 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,152 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,154 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T14:29:46,156 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:46,157 copying build/lib/lm_eval/tasks/include/default/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:46,159 copying build/lib/lm_eval/tasks/include/default/Georgian/_include_base_44_georgian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:46,161 copying build/lib/lm_eval/tasks/include/default/Georgian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:46,162 copying build/lib/lm_eval/tasks/include/default/Georgian/_georgian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T14:29:46,165 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,167 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,169 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,170 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,172 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,174 copying build/lib/lm_eval/tasks/include/default/Russian/_include_base_44_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,175 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,177 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,179 copying build/lib/lm_eval/tasks/include/default/Russian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,180 copying build/lib/lm_eval/tasks/include/default/Russian/_russian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,182 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,184 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T14:29:46,186 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Korean 2026-05-11T14:29:46,187 copying build/lib/lm_eval/tasks/include/default/Korean/_include_base_44_korean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T14:29:46,189 copying build/lib/lm_eval/tasks/include/default/Korean/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T14:29:46,191 copying build/lib/lm_eval/tasks/include/default/Korean/_korean_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T14:29:46,193 copying build/lib/lm_eval/tasks/include/default/Korean/include_base_44_korean_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T14:29:46,195 copying build/lib/lm_eval/tasks/include/default/Korean/include_base_44_korean_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T14:29:46,197 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,198 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,200 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,202 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,204 copying build/lib/lm_eval/tasks/include/default/Chinese/_chinese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,205 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,207 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,209 copying build/lib/lm_eval/tasks/include/default/Chinese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,211 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,213 copying build/lib/lm_eval/tasks/include/default/Chinese/_include_base_44_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,215 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,217 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T14:29:46,219 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:46,220 copying build/lib/lm_eval/tasks/include/default/Croatian/_croatian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:46,222 copying build/lib/lm_eval/tasks/include/default/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:46,224 copying build/lib/lm_eval/tasks/include/default/Croatian/include_base_44_croatian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:46,226 copying build/lib/lm_eval/tasks/include/default/Croatian/_include_base_44_croatian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:46,228 copying build/lib/lm_eval/tasks/include/default/Croatian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:46,230 copying build/lib/lm_eval/tasks/include/default/Croatian/include_base_44_croatian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T14:29:46,232 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,233 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,235 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,237 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,239 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,240 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,242 copying build/lib/lm_eval/tasks/include/default/Dutch/_dutch_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,244 copying build/lib/lm_eval/tasks/include/default/Dutch/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,245 copying build/lib/lm_eval/tasks/include/default/Dutch/_include_base_44_dutch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T14:29:46,248 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,249 copying build/lib/lm_eval/tasks/include/default/Hindi/_hindi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,251 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,252 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,254 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,255 copying build/lib/lm_eval/tasks/include/default/Hindi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,257 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,259 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,260 copying build/lib/lm_eval/tasks/include/default/Hindi/_include_base_44_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,262 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,264 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,266 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T14:29:46,268 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:46,269 copying build/lib/lm_eval/tasks/include/default/Tamil/_tamil_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:46,271 copying build/lib/lm_eval/tasks/include/default/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:46,273 copying build/lib/lm_eval/tasks/include/default/Tamil/include_base_44_tamil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:46,275 copying build/lib/lm_eval/tasks/include/default/Tamil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:46,276 copying build/lib/lm_eval/tasks/include/default/Tamil/_include_base_44_tamil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T14:29:46,279 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:46,280 copying build/lib/lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:46,281 copying build/lib/lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:46,283 copying build/lib/lm_eval/tasks/include/default/Hungarian/_include_base_44_hungarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:46,285 copying build/lib/lm_eval/tasks/include/default/Hungarian/_hungarian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:46,287 copying build/lib/lm_eval/tasks/include/default/Hungarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:46,288 copying build/lib/lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T14:29:46,290 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,291 copying build/lib/lm_eval/tasks/include/default/Turkish/_include_base_44_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,293 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,295 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,296 copying build/lib/lm_eval/tasks/include/default/Turkish/_turkish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,298 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,300 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,301 copying build/lib/lm_eval/tasks/include/default/Turkish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T14:29:46,304 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,305 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,307 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,308 copying build/lib/lm_eval/tasks/include/default/Greek/_greek_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,310 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,312 copying build/lib/lm_eval/tasks/include/default/Greek/_include_base_44_greek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,313 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,315 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,317 copying build/lib/lm_eval/tasks/include/default/Greek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,319 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,321 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T14:29:46,323 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,324 copying build/lib/lm_eval/tasks/include/default/Armenian/_include_base_44_armenian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,326 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,328 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,330 copying build/lib/lm_eval/tasks/include/default/Armenian/_armenian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,331 copying build/lib/lm_eval/tasks/include/default/Armenian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,333 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,334 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T14:29:46,337 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:46,338 copying build/lib/lm_eval/tasks/include/default/Hebrew/_hebrew_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:46,339 copying build/lib/lm_eval/tasks/include/default/Hebrew/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:46,341 copying build/lib/lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:46,343 copying build/lib/lm_eval/tasks/include/default/Hebrew/_include_base_44_hebrew.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:46,344 copying build/lib/lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T14:29:46,347 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/German 2026-05-11T14:29:46,348 copying build/lib/lm_eval/tasks/include/default/German/include_base_44_german_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T14:29:46,350 copying build/lib/lm_eval/tasks/include/default/German/include_base_44_german_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T14:29:46,352 copying build/lib/lm_eval/tasks/include/default/German/_german_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T14:29:46,353 copying build/lib/lm_eval/tasks/include/default/German/include_base_44_german_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T14:29:46,355 copying build/lib/lm_eval/tasks/include/default/German/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T14:29:46,357 copying build/lib/lm_eval/tasks/include/default/German/_include_base_44_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T14:29:46,360 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:46,361 copying build/lib/lm_eval/tasks/include/default/Japanese/include_base_44_japanese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:46,363 copying build/lib/lm_eval/tasks/include/default/Japanese/_japanese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:46,365 copying build/lib/lm_eval/tasks/include/default/Japanese/include_base_44_japanese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:46,366 copying build/lib/lm_eval/tasks/include/default/Japanese/include_base_44_japanese_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:46,368 copying build/lib/lm_eval/tasks/include/default/Japanese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:46,370 copying build/lib/lm_eval/tasks/include/default/Japanese/_include_base_44_japanese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T14:29:46,372 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Basque 2026-05-11T14:29:46,373 copying build/lib/lm_eval/tasks/include/default/Basque/_basque_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T14:29:46,375 copying build/lib/lm_eval/tasks/include/default/Basque/include_base_44_basque_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T14:29:46,377 copying build/lib/lm_eval/tasks/include/default/Basque/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T14:29:46,379 copying build/lib/lm_eval/tasks/include/default/Basque/_include_base_44_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T14:29:46,381 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:46,382 copying build/lib/lm_eval/tasks/include/default/Bulgarian/_include_base_44_bulgarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:46,383 copying build/lib/lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:46,385 copying build/lib/lm_eval/tasks/include/default/Bulgarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:46,387 copying build/lib/lm_eval/tasks/include/default/Bulgarian/_bulgarian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:46,389 copying build/lib/lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:46,391 copying build/lib/lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T14:29:46,394 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og 2026-05-11T14:29:46,396 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,397 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/_indonesian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,399 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,401 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,403 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,405 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,407 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,409 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/_include_base_44_indonesian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,411 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T14:29:46,413 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:46,414 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/_tagalog_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:46,416 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:46,417 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/_include_base_44_tagalog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:46,419 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:46,421 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T14:29:46,424 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,425 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,427 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/_north macedonian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,430 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,431 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,433 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,435 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/_include_base_44_north macedonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,437 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T14:29:46,440 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,441 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,443 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,445 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,447 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,449 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/_malayalam_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,450 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,452 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,454 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,456 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/_include_base_44_malayalam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T14:29:46,459 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:46,461 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:46,463 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/_belarusian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:46,465 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:46,467 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:46,469 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/_include_base_44_belarusian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:46,470 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T14:29:46,473 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:46,474 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/_kazakh_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:46,476 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:46,478 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:46,479 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/_include_base_44_kazakh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T14:29:46,482 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,483 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,485 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/_include_base_44_persian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,486 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,488 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,490 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/_persian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,492 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,493 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,495 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T14:29:46,497 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,498 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,500 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,502 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/_arabic_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,503 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,505 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,507 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/_include_base_44_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,509 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,511 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,512 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T14:29:46,515 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:46,516 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/_ukrainian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:46,518 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:46,519 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/_include_base_44_ukrainian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:46,521 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:46,523 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:46,525 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T14:29:46,527 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,528 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,530 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,532 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,534 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,535 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,537 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/_include_base_44_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,539 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/_telugu_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T14:29:46,541 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:46,542 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/_include_base_44_urdu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:46,544 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/_urdu_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:46,546 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:46,548 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:46,550 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:46,552 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T14:29:46,554 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,555 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,558 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/_include_base_44_lithuanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,559 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,561 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/_lithuanian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,563 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,565 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,567 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,569 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T14:29:46,571 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:46,572 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/_include_base_44_vietnamese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:46,574 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:46,576 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:46,578 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/_vietnamese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:46,580 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:46,582 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T14:29:46,584 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,585 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/_albanian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,587 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,589 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,590 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,592 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,594 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,596 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,598 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/_include_base_44_albanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T14:29:46,600 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,601 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,603 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/_finnish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,604 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,606 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,608 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,610 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,612 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,613 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/_include_base_44_finnish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T14:29:46,616 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,617 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/_spanish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,619 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,621 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,622 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,624 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,626 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/_include_base_44_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,628 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T14:29:46,630 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:46,631 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:46,633 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/_include_base_44_nepali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:46,635 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/_nepali_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:46,637 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:46,639 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T14:29:46,641 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:46,642 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:46,644 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/_include_base_44_polish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:46,647 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:46,649 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/_polish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:46,651 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:46,653 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T14:29:46,655 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,656 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/_azerbaijani_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,658 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,659 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,661 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,663 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,665 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,666 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,668 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,670 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T14:29:46,672 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:46,673 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:46,675 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:46,677 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/_include_base_44_serbian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:46,679 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/_serbian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:46,680 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:46,683 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T14:29:46,685 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,686 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,688 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/_include_base_44_portuguese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,690 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,691 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,693 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,695 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,697 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,699 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/_portuguese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,700 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T14:29:46,703 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,704 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,706 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,707 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,709 copying build/lib/lm_eval/tasks/include/few_shot_og/French/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,711 copying build/lib/lm_eval/tasks/include/few_shot_og/French/_include_base_44_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,713 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,715 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,717 copying build/lib/lm_eval/tasks/include/few_shot_og/French/_french_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T14:29:46,719 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,720 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,723 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/_uzbek_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,724 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,726 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,728 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,729 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/_include_base_44_uzbek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,731 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T14:29:46,733 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:46,734 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:46,736 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:46,738 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:46,740 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:46,742 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/_malay_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:46,743 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/_include_base_44_malay.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T14:29:46,746 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,747 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/_italian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,749 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,751 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,752 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,754 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,756 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/_include_base_44_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,758 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,759 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,761 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T14:29:46,764 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,765 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,767 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,769 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,770 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,772 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,774 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/_estonian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,776 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/_include_base_44_estonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,778 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T14:29:46,780 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,781 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,783 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/_bengali_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,785 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/_include_base_44_bengali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,788 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,790 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,792 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,794 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T14:29:46,797 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:46,798 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:46,800 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/_include_base_44_georgian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:46,803 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:46,805 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/_georgian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T14:29:46,807 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,808 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/_russian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,810 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,811 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,813 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,815 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,817 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/_include_base_44_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,818 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,820 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,822 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,824 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,825 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T14:29:46,828 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:46,829 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/_include_base_44_korean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:46,831 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/_korean_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:46,833 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:46,834 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:46,836 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T14:29:46,839 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,840 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,842 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/_chinese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,843 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,845 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,847 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,848 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,850 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,852 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,853 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/_include_base_44_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,855 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,857 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T14:29:46,859 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:46,860 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:46,862 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:46,864 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/_croatian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:46,866 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/_include_base_44_croatian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:46,867 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:46,869 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T14:29:46,872 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,873 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,875 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,877 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,878 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,880 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,882 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,884 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/_dutch_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,885 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/_include_base_44_dutch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T14:29:46,888 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,889 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,891 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/_hindi_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,892 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,894 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,897 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,898 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,900 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,902 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/_include_base_44_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,904 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,905 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,907 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T14:29:46,910 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:46,911 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:46,912 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/_tamil_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:46,914 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:46,916 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:46,918 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/_include_base_44_tamil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T14:29:46,920 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:46,921 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:46,923 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:46,925 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/_include_base_44_hungarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:46,927 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/_hungarian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:46,928 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:46,930 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T14:29:46,932 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,933 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/_include_base_44_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,936 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,937 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,939 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,941 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,943 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,944 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/_turkish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T14:29:46,947 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,948 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,950 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,952 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,953 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/_include_base_44_greek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,955 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,957 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,959 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,961 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,963 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/_greek_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,964 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T14:29:46,967 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,968 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/_include_base_44_armenian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,970 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,972 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,974 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/_armenian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,976 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,977 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,979 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T14:29:46,981 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:46,982 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/_hebrew_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:46,985 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:46,986 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:46,988 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/_include_base_44_hebrew.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:46,990 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T14:29:46,992 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:46,994 copying build/lib/lm_eval/tasks/include/few_shot_og/German/include_base_44_german_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:46,995 copying build/lib/lm_eval/tasks/include/few_shot_og/German/include_base_44_german_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:46,997 copying build/lib/lm_eval/tasks/include/few_shot_og/German/include_base_44_german_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:46,999 copying build/lib/lm_eval/tasks/include/few_shot_og/German/_german_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:47,001 copying build/lib/lm_eval/tasks/include/few_shot_og/German/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:47,002 copying build/lib/lm_eval/tasks/include/few_shot_og/German/_include_base_44_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T14:29:47,005 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:47,006 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:47,008 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/_japanese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:47,010 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:47,011 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:47,013 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:47,015 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/_include_base_44_japanese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T14:29:47,017 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:47,018 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/include_base_44_basque_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:47,020 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:47,022 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/_basque_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:47,024 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/_include_base_44_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T14:29:47,027 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:47,028 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/_include_base_44_bulgarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:47,031 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:47,034 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/_bulgarian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:47,036 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:47,038 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:47,041 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T14:29:47,045 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2026-05-11T14:29:47,046 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,048 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,050 copying build/lib/lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,052 copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,053 copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,055 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,057 copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,059 copying build/lib/lm_eval/tasks/blimp/npi_present_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,060 copying build/lib/lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,062 copying build/lib/lm_eval/tasks/blimp/npi_present_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,065 copying build/lib/lm_eval/tasks/blimp/causative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,068 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,069 copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,071 copying build/lib/lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,073 copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,075 copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,076 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,078 copying build/lib/lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,080 copying build/lib/lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,082 copying build/lib/lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,084 copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,086 copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,088 copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,090 copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,091 copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,093 copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,095 copying build/lib/lm_eval/tasks/blimp/adjunct_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,097 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,098 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,100 copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,102 copying build/lib/lm_eval/tasks/blimp/only_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,104 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,106 copying build/lib/lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,107 copying build/lib/lm_eval/tasks/blimp/generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,110 copying build/lib/lm_eval/tasks/blimp/drop_argument.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,112 copying build/lib/lm_eval/tasks/blimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,114 copying build/lib/lm_eval/tasks/blimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,117 copying build/lib/lm_eval/tasks/blimp/complex_NP_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,120 copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,122 copying build/lib/lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,124 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,126 copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,127 copying build/lib/lm_eval/tasks/blimp/passive_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,129 copying build/lib/lm_eval/tasks/blimp/wh_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,131 copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,132 copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,134 copying build/lib/lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,136 copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,138 copying build/lib/lm_eval/tasks/blimp/inchoative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,140 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,141 copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,143 copying build/lib/lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,145 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,147 copying build/lib/lm_eval/tasks/blimp/_blimp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,149 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,150 copying build/lib/lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,152 copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,154 copying build/lib/lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,155 copying build/lib/lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,157 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,159 copying build/lib/lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,161 copying build/lib/lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,162 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,164 copying build/lib/lm_eval/tasks/blimp/intransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,166 copying build/lib/lm_eval/tasks/blimp/passive_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,168 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,169 copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,171 copying build/lib/lm_eval/tasks/blimp/transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,173 copying build/lib/lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,175 copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,176 copying build/lib/lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T14:29:47,179 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmmu 2026-05-11T14:29:47,180 copying build/lib/lm_eval/tasks/mmmu/mmmu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,182 copying build/lib/lm_eval/tasks/mmmu/mmmu_art_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,184 copying build/lib/lm_eval/tasks/mmmu/_tech_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,186 copying build/lib/lm_eval/tasks/mmmu/mmmu_music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,188 copying build/lib/lm_eval/tasks/mmmu/mmmu_basic_medical_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,189 copying build/lib/lm_eval/tasks/mmmu/mmmu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,191 copying build/lib/lm_eval/tasks/mmmu/mmmu_materials.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,193 copying build/lib/lm_eval/tasks/mmmu/mmmu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,195 copying build/lib/lm_eval/tasks/mmmu/mmmu_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,197 copying build/lib/lm_eval/tasks/mmmu/mmmu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,198 copying build/lib/lm_eval/tasks/mmmu/mmmu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,200 copying build/lib/lm_eval/tasks/mmmu/mmmu_electronics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,201 copying build/lib/lm_eval/tasks/mmmu/mmmu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,203 copying build/lib/lm_eval/tasks/mmmu/_health_and_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,205 copying build/lib/lm_eval/tasks/mmmu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,207 copying build/lib/lm_eval/tasks/mmmu/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,209 copying build/lib/lm_eval/tasks/mmmu/_art_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,210 copying build/lib/lm_eval/tasks/mmmu/mmmu_finance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,212 copying build/lib/lm_eval/tasks/mmmu/_humanities_and_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,214 copying build/lib/lm_eval/tasks/mmmu/_mmmu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,216 copying build/lib/lm_eval/tasks/mmmu/mmmu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,218 copying build/lib/lm_eval/tasks/mmmu/mmmu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,219 copying build/lib/lm_eval/tasks/mmmu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,222 copying build/lib/lm_eval/tasks/mmmu/mmmu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,224 copying build/lib/lm_eval/tasks/mmmu/mmmu_architecture_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,226 copying build/lib/lm_eval/tasks/mmmu/mmmu_diagnostics_and_laboratory_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,228 copying build/lib/lm_eval/tasks/mmmu/mmmu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,230 copying build/lib/lm_eval/tasks/mmmu/mmmu_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,233 copying build/lib/lm_eval/tasks/mmmu/_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,235 copying build/lib/lm_eval/tasks/mmmu/mmmu_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,237 copying build/lib/lm_eval/tasks/mmmu/_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,238 copying build/lib/lm_eval/tasks/mmmu/mmmu_art.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,240 copying build/lib/lm_eval/tasks/mmmu/mmmu_energy_and_power.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,243 copying build/lib/lm_eval/tasks/mmmu/mmmu_manage.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,245 copying build/lib/lm_eval/tasks/mmmu/mmmu_public_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,247 copying build/lib/lm_eval/tasks/mmmu/mmmu_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,250 copying build/lib/lm_eval/tasks/mmmu/mmmu_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,252 copying build/lib/lm_eval/tasks/mmmu/mmmu_pharmacy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,254 copying build/lib/lm_eval/tasks/mmmu/mmmu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,257 copying build/lib/lm_eval/tasks/mmmu/mmmu_agriculture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T14:29:47,260 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmlu 2026-05-11T14:29:47,261 copying build/lib/lm_eval/tasks/tmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu 2026-05-11T14:29:47,263 copying build/lib/lm_eval/tasks/tmlu/subject.tsv -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu 2026-05-11T14:29:47,267 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,268 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,270 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_driving_rule.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,273 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_clinical_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,275 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_earth_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,277 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,279 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,281 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_clinical_psychologist.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,283 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,285 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,287 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,289 copying build/lib/lm_eval/tasks/tmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,291 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_basic_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,293 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_tour_leader.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,295 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_nutritionist.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,297 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_earth_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,299 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,301 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,303 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,305 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,307 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,309 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,311 copying build/lib/lm_eval/tasks/tmlu/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,313 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,315 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_tour_guide.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,317 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,319 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_accountant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,321 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,323 copying build/lib/lm_eval/tasks/tmlu/default/_tmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,325 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,327 copying build/lib/lm_eval/tasks/tmlu/default/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,330 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,332 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_taiwan_tourist_resources.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,334 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_lawyer_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,336 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,338 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T14:29:47,341 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/egymmlu 2026-05-11T14:29:47,342 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,345 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_driving_test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,568 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,571 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_philosophy_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,574 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,577 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,580 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_political_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,583 copying build/lib/lm_eval/tasks/egymmlu/_default_egymmlu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,586 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,589 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,592 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,595 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,598 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_natural_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,601 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,603 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,607 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,609 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,612 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,615 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,619 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,622 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,624 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,627 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_arabic_language_(grammar).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,630 copying build/lib/lm_eval/tasks/egymmlu/_egymmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,633 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,636 copying build/lib/lm_eval/tasks/egymmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,639 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,642 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,645 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,647 copying build/lib/lm_eval/tasks/egymmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,649 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,651 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,653 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,654 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,656 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_management_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,658 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,660 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_arabic_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,661 copying build/lib/lm_eval/tasks/egymmlu/_egymmlu_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,663 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,665 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,667 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,668 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,670 copying build/lib/lm_eval/tasks/egymmlu/_egymmlu_ar_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,672 copying build/lib/lm_eval/tasks/egymmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,673 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_arabic_language_(general).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,675 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,677 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,678 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,680 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,682 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,683 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T14:29:47,686 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/moral_stories 2026-05-11T14:29:47,687 copying build/lib/lm_eval/tasks/moral_stories/moral_stories.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/moral_stories 2026-05-11T14:29:47,688 copying build/lib/lm_eval/tasks/moral_stories/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/moral_stories 2026-05-11T14:29:47,690 copying build/lib/lm_eval/tasks/moral_stories/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/moral_stories 2026-05-11T14:29:47,693 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mastermind 2026-05-11T14:29:47,694 copying build/lib/lm_eval/tasks/mastermind/mastermind_46_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T14:29:47,696 copying build/lib/lm_eval/tasks/mastermind/mastermind_46_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T14:29:47,697 copying build/lib/lm_eval/tasks/mastermind/mastermind_35_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T14:29:47,699 copying build/lib/lm_eval/tasks/mastermind/mastermind_24_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T14:29:47,701 copying build/lib/lm_eval/tasks/mastermind/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T14:29:47,703 copying build/lib/lm_eval/tasks/mastermind/mastermind_35_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T14:29:47,704 copying build/lib/lm_eval/tasks/mastermind/mastermind_24_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T14:29:47,707 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,708 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10cm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,710 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,712 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,714 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_atc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,715 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,717 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,718 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,720 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9cm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,722 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,723 copying build/lib/lm_eval/tasks/med_concepts_qa/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,725 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,726 copying build/lib/lm_eval/tasks/med_concepts_qa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,728 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,730 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,732 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9proc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,733 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,735 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,737 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10proc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,738 copying build/lib/lm_eval/tasks/med_concepts_qa/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,740 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,742 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,744 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,745 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,747 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T14:29:47,750 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/babilong 2026-05-11T14:29:47,751 copying build/lib/lm_eval/tasks/babilong/babilong_qa3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,753 copying build/lib/lm_eval/tasks/babilong/babilong_qa7.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,755 copying build/lib/lm_eval/tasks/babilong/common_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,757 copying build/lib/lm_eval/tasks/babilong/babilong_qa16.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,759 copying build/lib/lm_eval/tasks/babilong/babilong_qa5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,760 copying build/lib/lm_eval/tasks/babilong/babilong_qa12.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,762 copying build/lib/lm_eval/tasks/babilong/babilong_qa1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,764 copying build/lib/lm_eval/tasks/babilong/babilong_qa18.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,766 copying build/lib/lm_eval/tasks/babilong/babilong_longctx.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,768 copying build/lib/lm_eval/tasks/babilong/babilong_qa14.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,769 copying build/lib/lm_eval/tasks/babilong/babilong_qa10.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,771 copying build/lib/lm_eval/tasks/babilong/babilong_qa2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,773 copying build/lib/lm_eval/tasks/babilong/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,775 copying build/lib/lm_eval/tasks/babilong/babilong_qa8.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,777 copying build/lib/lm_eval/tasks/babilong/_babilong_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,778 copying build/lib/lm_eval/tasks/babilong/babilong_qa17.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,780 copying build/lib/lm_eval/tasks/babilong/babilong_qa11.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,782 copying build/lib/lm_eval/tasks/babilong/babilong_qa9.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,784 copying build/lib/lm_eval/tasks/babilong/babilong.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,785 copying build/lib/lm_eval/tasks/babilong/babilong_qa20.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,787 copying build/lib/lm_eval/tasks/babilong/babilong_qa15.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,789 copying build/lib/lm_eval/tasks/babilong/babilong_qa4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,791 copying build/lib/lm_eval/tasks/babilong/babilong_qa19.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,792 copying build/lib/lm_eval/tasks/babilong/babilong_qa13.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,794 copying build/lib/lm_eval/tasks/babilong/babilong_qa6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T14:29:47,797 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/longbench2 2026-05-11T14:29:47,798 copying build/lib/lm_eval/tasks/longbench2/legal_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,800 copying build/lib/lm_eval/tasks/longbench2/many_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,801 copying build/lib/lm_eval/tasks/longbench2/agent_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,803 copying build/lib/lm_eval/tasks/longbench2/news_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,805 copying build/lib/lm_eval/tasks/longbench2/user_guide.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,806 copying build/lib/lm_eval/tasks/longbench2/govt_multi_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,808 copying build/lib/lm_eval/tasks/longbench2/academic_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,810 copying build/lib/lm_eval/tasks/longbench2/lit_single_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,811 copying build/lib/lm_eval/tasks/longbench2/academic_multi_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,813 copying build/lib/lm_eval/tasks/longbench2/_longbench_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,815 copying build/lib/lm_eval/tasks/longbench2/_longbench2_incontext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,816 copying build/lib/lm_eval/tasks/longbench2/_longbench2_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,818 copying build/lib/lm_eval/tasks/longbench2/_longbench2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,820 copying build/lib/lm_eval/tasks/longbench2/graph.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,821 copying build/lib/lm_eval/tasks/longbench2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,823 copying build/lib/lm_eval/tasks/longbench2/govt_single_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,825 copying build/lib/lm_eval/tasks/longbench2/_longbench2_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,827 copying build/lib/lm_eval/tasks/longbench2/legal_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,828 copying build/lib/lm_eval/tasks/longbench2/table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,830 copying build/lib/lm_eval/tasks/longbench2/dialogue_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,832 copying build/lib/lm_eval/tasks/longbench2/_longbench2_structured.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,833 copying build/lib/lm_eval/tasks/longbench2/fin_single_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,835 copying build/lib/lm_eval/tasks/longbench2/translate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,837 copying build/lib/lm_eval/tasks/longbench2/detective.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,839 copying build/lib/lm_eval/tasks/longbench2/_longbench2_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,840 copying build/lib/lm_eval/tasks/longbench2/fin_multi_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,842 copying build/lib/lm_eval/tasks/longbench2/longbench2_code.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,844 copying build/lib/lm_eval/tasks/longbench2/event_order.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T14:29:47,846 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue 2026-05-11T14:29:47,847 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,849 copying build/lib/lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,851 copying build/lib/lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,852 copying build/lib/lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,854 copying build/lib/lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,856 copying build/lib/lm_eval/tasks/code_x_glue/code-text/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,857 copying build/lib/lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,859 copying build/lib/lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,861 copying build/lib/lm_eval/tasks/code_x_glue/code-text/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,863 copying build/lib/lm_eval/tasks/code_x_glue/code-text/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,864 copying build/lib/lm_eval/tasks/code_x_glue/code-text/_codexglue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,866 copying build/lib/lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T14:29:47,868 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq 2026-05-11T14:29:47,869 copying build/lib/lm_eval/tasks/sciq/sciq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/sciq 2026-05-11T14:29:47,871 copying build/lib/lm_eval/tasks/sciq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/sciq 2026-05-11T14:29:47,874 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,875 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,876 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,878 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,880 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_winogrande.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,881 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,883 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,885 copying build/lib/lm_eval/tasks/tinyBenchmarks/agg_functions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,887 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,889 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,891 copying build/lib/lm_eval/tasks/tinyBenchmarks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,893 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,894 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyArc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,896 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T14:29:47,898 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/chartqa 2026-05-11T14:29:47,899 copying build/lib/lm_eval/tasks/chartqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T14:29:47,901 copying build/lib/lm_eval/tasks/chartqa/chartqa_llama_90.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T14:29:47,903 copying build/lib/lm_eval/tasks/chartqa/chartqa_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T14:29:47,905 copying build/lib/lm_eval/tasks/chartqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T14:29:47,907 copying build/lib/lm_eval/tasks/chartqa/chartqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T14:29:47,909 copying build/lib/lm_eval/tasks/_yaml_loader.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T14:29:47,911 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2026-05-11T14:29:47,912 copying build/lib/lm_eval/tasks/minerva_math/minerva_math500.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,914 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,916 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,917 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,919 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,921 copying build/lib/lm_eval/tasks/minerva_math/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,923 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,924 copying build/lib/lm_eval/tasks/minerva_math/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,927 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,929 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T14:29:47,931 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3 2026-05-11T14:29:47,933 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct 2026-05-11T14:29:47,935 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,937 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,939 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,941 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,942 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,944 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,946 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,948 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,950 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,951 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,953 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,955 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,957 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,960 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,962 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,963 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,965 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,967 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,969 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,971 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,973 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,975 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,977 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,979 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,981 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,983 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,985 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,987 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,989 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,992 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,994 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,996 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,998 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:47,999 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,001 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,003 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,005 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,007 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,009 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,011 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,013 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,015 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,017 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,019 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,021 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,023 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,025 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,027 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,029 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,031 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,033 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,035 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,037 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,039 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,041 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,043 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,044 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,046 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,048 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,050 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,052 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,054 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,056 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,058 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,060 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T14:29:48,063 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,065 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,067 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,071 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,073 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,075 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,077 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,079 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,081 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,084 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,086 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,089 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,091 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,093 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,095 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,097 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,099 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,102 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,104 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,106 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,109 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,111 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,113 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,115 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,118 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,121 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,123 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,125 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,127 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,130 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,132 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,134 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,136 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,138 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,141 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,143 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,145 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,147 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,150 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,152 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,154 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,157 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,159 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,161 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,163 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,165 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,167 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,170 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,172 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,175 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,177 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,179 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,182 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,184 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,186 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,190 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,192 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,194 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,196 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,199 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,201 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,203 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,205 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,208 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,210 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T14:29:48,213 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,214 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,217 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,219 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,220 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,222 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,224 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,226 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,228 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,230 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,232 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,234 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,236 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,238 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,239 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,241 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,243 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,245 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,247 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,248 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,250 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,252 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,254 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,256 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,258 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,260 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,262 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,263 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,266 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,268 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,270 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,272 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,274 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,276 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,278 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,279 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,281 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,283 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,285 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,287 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,289 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,291 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,293 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,294 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,297 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,299 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,301 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,302 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,304 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,306 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,308 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,310 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,312 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,314 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,316 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,318 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,320 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,322 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,323 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,326 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,328 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,330 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,332 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,333 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,335 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T14:29:48,338 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,340 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,342 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,343 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,345 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,347 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,349 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,351 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,353 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,355 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,358 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,360 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,362 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,364 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,366 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,368 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,371 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,373 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,374 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,376 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,378 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,379 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,381 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,383 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,385 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,387 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,389 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,391 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,393 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,395 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,397 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,399 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,401 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,403 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,405 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,407 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,409 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,411 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,413 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,415 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,417 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,418 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,420 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,422 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,423 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,425 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,427 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,429 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,431 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,433 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,434 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,436 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,438 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,440 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,442 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,444 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,446 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,448 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,450 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,451 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,454 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,455 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,457 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,459 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,461 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T14:29:48,464 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,465 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,467 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,469 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,471 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,473 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,475 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,477 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,479 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,481 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,483 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,484 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,486 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,488 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,491 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,493 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,495 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,497 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,498 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,500 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,502 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,504 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,505 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,507 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,509 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,511 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,513 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,515 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,516 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,518 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,520 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,522 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,524 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,526 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,528 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,530 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,532 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,534 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,536 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,537 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,539 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,542 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,544 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,546 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,548 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,551 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,554 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,556 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,559 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,561 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,563 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,565 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,567 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,570 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,571 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,574 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,576 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,578 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,580 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,582 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,584 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,586 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,588 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,590 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,592 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T14:29:48,595 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,596 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,598 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,600 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,601 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,603 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,605 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,607 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,609 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,610 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,612 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,614 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,616 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,618 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,620 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,621 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,623 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,625 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,627 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,629 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,630 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,632 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,634 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,635 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,637 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,639 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,640 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,642 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,644 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,646 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,648 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,650 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,652 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,654 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,655 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,657 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,659 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,661 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,663 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,664 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,666 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,668 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,670 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,671 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,673 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,675 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,676 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,678 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,680 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,681 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,683 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,685 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,687 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,689 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,691 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,692 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,694 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,696 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,698 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,699 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,701 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,703 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,705 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,706 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T14:29:48,709 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T14:29:48,710 copying build/lib/lm_eval/tasks/llama3/instruct/arc_challenge/arc_challenge_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T14:29:48,712 copying build/lib/lm_eval/tasks/llama3/instruct/arc_challenge/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T14:29:48,715 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,716 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,718 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,719 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,721 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,722 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,724 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,726 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,728 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,730 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,732 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,734 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,735 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,737 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,739 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,741 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,744 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,746 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,748 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,750 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,752 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,754 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,755 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,757 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,759 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,761 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,763 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,765 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,767 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,769 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,771 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,773 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,775 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,777 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,779 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,781 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,783 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,785 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,787 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,789 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,791 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,793 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,795 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,796 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,798 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,800 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,802 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,804 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,806 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,808 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,809 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,811 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,813 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,815 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,817 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,819 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,821 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,823 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,825 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,827 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,829 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,831 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,833 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,835 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,837 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T14:29:48,840 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T14:29:48,841 copying build/lib/lm_eval/tasks/llama3/instruct/gsm8k/gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T14:29:48,844 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,845 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,847 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,849 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,851 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,853 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,855 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,857 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,859 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,861 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,863 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,865 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,867 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,869 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,871 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,873 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,874 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,876 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,878 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,880 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,881 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,883 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,885 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,886 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,888 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,890 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,892 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,894 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,896 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,897 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,900 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,901 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,903 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,904 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,906 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,908 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,910 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,912 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,914 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,916 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,917 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,919 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,921 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,923 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,925 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,926 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,928 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,930 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,932 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,934 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,935 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,937 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,939 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,941 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,942 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,944 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,946 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,948 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,950 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,952 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,954 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,955 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,957 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,959 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T14:29:48,962 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,963 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,965 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,967 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,968 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,970 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,972 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,974 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,975 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,977 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,979 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,981 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,983 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,986 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,988 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,991 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,994 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,996 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T14:29:48,999 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,000 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,002 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,004 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,007 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,009 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,011 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,013 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,016 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,018 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,020 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,023 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,024 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,027 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,030 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,032 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,034 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,037 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,039 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,042 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,044 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,047 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,049 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,051 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,053 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,056 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,058 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,061 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,063 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,066 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,068 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,071 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,073 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,075 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,078 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,080 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,082 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,085 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,087 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,090 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,092 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,094 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,096 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,099 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,101 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,103 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,105 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,107 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,110 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,112 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,114 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,116 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,118 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,121 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,124 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,125 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,128 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,129 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,132 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,134 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,136 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,139 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,142 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,144 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,146 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T14:29:49,149 copying build/lib/lm_eval/tasks/llama3/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3 2026-05-11T14:29:49,151 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/commonsense_qa 2026-05-11T14:29:49,152 copying build/lib/lm_eval/tasks/commonsense_qa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/commonsense_qa 2026-05-11T14:29:49,154 copying build/lib/lm_eval/tasks/commonsense_qa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/commonsense_qa 2026-05-11T14:29:49,160 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,161 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_prepP.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,163 copying build/lib/lm_eval/tasks/zhoblimp/BA_verb_le_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,165 copying build/lib/lm_eval/tasks/zhoblimp/preposition_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,167 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_intran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,169 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_wh_question_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,171 copying build/lib/lm_eval/tasks/zhoblimp/existential_there_subject_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,172 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_dou.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,174 copying build/lib/lm_eval/tasks/zhoblimp/BA_duplicate_argument.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,176 copying build/lib/lm_eval/tasks/zhoblimp/question_particle_daodi_choice_intran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,177 copying build/lib/lm_eval/tasks/zhoblimp/ellipsis_double_object.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,179 copying build/lib/lm_eval/tasks/zhoblimp/BA_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,181 copying build/lib/lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,183 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_scope_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,184 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_1_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,186 copying build/lib/lm_eval/tasks/zhoblimp/BEI_construction_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,188 copying build/lib/lm_eval/tasks/zhoblimp/noun_phrase_conjunction_jian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,190 copying build/lib/lm_eval/tasks/zhoblimp/agent_causative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,192 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_d.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,194 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_c_command_number.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,195 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,197 copying build/lib/lm_eval/tasks/zhoblimp/question_particle_daodi_choice_tran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,199 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,201 copying build/lib/lm_eval/tasks/zhoblimp/BA_verb_le_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,202 copying build/lib/lm_eval/tasks/zhoblimp/adjective_transitive_dui.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,204 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,206 copying build/lib/lm_eval/tasks/zhoblimp/BA_BEI_subj_drop.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,208 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,210 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_1_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,212 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_ruguo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,213 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_locP.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,216 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,218 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_scope_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,219 copying build/lib/lm_eval/tasks/zhoblimp/preposition_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,221 copying build/lib/lm_eval/tasks/zhoblimp/intransitive_double_obj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,223 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,225 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_SOV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,226 copying build/lib/lm_eval/tasks/zhoblimp/ya_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,228 copying build/lib/lm_eval/tasks/zhoblimp/relative_operator_who.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,230 copying build/lib/lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,232 copying build/lib/lm_eval/tasks/zhoblimp/passive_no_adj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,233 copying build/lib/lm_eval/tasks/zhoblimp/anaphor_gender_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,235 copying build/lib/lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,237 copying build/lib/lm_eval/tasks/zhoblimp/passive_intransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,239 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_domain.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,241 copying build/lib/lm_eval/tasks/zhoblimp/causative_shi_ba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,243 copying build/lib/lm_eval/tasks/zhoblimp/plural_cardinal_men_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,244 copying build/lib/lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,247 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_suoyou.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,249 copying build/lib/lm_eval/tasks/zhoblimp/BA_inversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,251 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,253 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_wh_question_obj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,255 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,257 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_tran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,259 copying build/lib/lm_eval/tasks/zhoblimp/BEI_preposition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,261 copying build/lib/lm_eval/tasks/zhoblimp/classifier_noun_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,263 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,265 copying build/lib/lm_eval/tasks/zhoblimp/intransitive_no_obj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,267 copying build/lib/lm_eval/tasks/zhoblimp/right_yijing_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,269 copying build/lib/lm_eval/tasks/zhoblimp/PN_numP_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,271 copying build/lib/lm_eval/tasks/zhoblimp/zhoblimp_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,274 copying build/lib/lm_eval/tasks/zhoblimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,276 copying build/lib/lm_eval/tasks/zhoblimp/relative_operator_intepretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,278 copying build/lib/lm_eval/tasks/zhoblimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,280 copying build/lib/lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,282 copying build/lib/lm_eval/tasks/zhoblimp/BEI_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,284 copying build/lib/lm_eval/tasks/zhoblimp/right_yijing_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,286 copying build/lib/lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,288 copying build/lib/lm_eval/tasks/zhoblimp/modal_raising_topicalization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,290 copying build/lib/lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,292 copying build/lib/lm_eval/tasks/zhoblimp/BA_meiba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,294 copying build/lib/lm_eval/tasks/zhoblimp/BA_no_progressive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,297 copying build/lib/lm_eval/tasks/zhoblimp/passive_body_part.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,299 copying build/lib/lm_eval/tasks/zhoblimp/question_V_not_VP_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,301 copying build/lib/lm_eval/tasks/zhoblimp/control_modal_vs_raising_modal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,303 copying build/lib/lm_eval/tasks/zhoblimp/you_yige.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,305 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A_indirect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,308 copying build/lib/lm_eval/tasks/zhoblimp/PN_numP_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,310 copying build/lib/lm_eval/tasks/zhoblimp/BA_suo_adverbial_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,312 copying build/lib/lm_eval/tasks/zhoblimp/BA_no_stative_verb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,314 copying build/lib/lm_eval/tasks/zhoblimp/modal_raising_hui.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,316 copying build/lib/lm_eval/tasks/zhoblimp/BEI_construction_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,318 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_domain_number.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,320 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_A_not_A_question.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,322 copying build/lib/lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,325 copying build/lib/lm_eval/tasks/zhoblimp/verb_negation_particle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,327 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A_daodi_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,329 copying build/lib/lm_eval/tasks/zhoblimp/agent_animacy_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,330 copying build/lib/lm_eval/tasks/zhoblimp/agent_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,332 copying build/lib/lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,335 copying build/lib/lm_eval/tasks/zhoblimp/classifier_noun_agreement_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,336 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,338 copying build/lib/lm_eval/tasks/zhoblimp/ellipsis_adj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,340 copying build/lib/lm_eval/tasks/zhoblimp/left_dou.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,342 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A_daodi_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,344 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,346 copying build/lib/lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,348 copying build/lib/lm_eval/tasks/zhoblimp/nominal_definite_men.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,351 copying build/lib/lm_eval/tasks/zhoblimp/agent_animacy_adv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,353 copying build/lib/lm_eval/tasks/zhoblimp/anaphor_number_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,355 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,357 copying build/lib/lm_eval/tasks/zhoblimp/passive_suo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,359 copying build/lib/lm_eval/tasks/zhoblimp/BA_suo_adverbial_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,361 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_long_left.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,363 copying build/lib/lm_eval/tasks/zhoblimp/question_particle_nandao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,365 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,367 copying build/lib/lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,370 copying build/lib/lm_eval/tasks/zhoblimp/classifier_noun_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,372 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_c_command.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,374 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,376 copying build/lib/lm_eval/tasks/zhoblimp/question_V_not_VP_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,378 copying build/lib/lm_eval/tasks/zhoblimp/nominal_modal_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,380 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_OSV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,382 copying build/lib/lm_eval/tasks/zhoblimp/plural_cardinal_men_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,384 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_short.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,386 copying build/lib/lm_eval/tasks/zhoblimp/agent_animacy_passive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,388 copying build/lib/lm_eval/tasks/zhoblimp/BA_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,390 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,392 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,394 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_conditional.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,396 copying build/lib/lm_eval/tasks/zhoblimp/noun_adjective_shi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,399 copying build/lib/lm_eval/tasks/zhoblimp/ellipsis_n_bar_class.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T14:29:49,401 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard 2026-05-11T14:29:49,403 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:49,404 copying build/lib/lm_eval/tasks/leaderboard/musr/musr_team_allocation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:49,407 copying build/lib/lm_eval/tasks/leaderboard/musr/musr_object_placements.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:49,409 copying build/lib/lm_eval/tasks/leaderboard/musr/musr_murder_mysteries.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:49,411 copying build/lib/lm_eval/tasks/leaderboard/musr/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:49,414 copying build/lib/lm_eval/tasks/leaderboard/musr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:49,416 copying build/lib/lm_eval/tasks/leaderboard/musr/_musr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T14:29:49,419 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,420 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,423 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,425 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,427 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,430 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,432 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,435 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,437 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,439 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,442 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,444 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,446 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,448 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,450 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,452 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,455 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,457 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,459 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,461 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,463 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,465 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,467 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,470 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,472 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,474 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,477 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/_leaderboard_bbh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T14:29:49,479 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:49,481 copying build/lib/lm_eval/tasks/leaderboard/gpqa/gpqa_main_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:49,483 copying build/lib/lm_eval/tasks/leaderboard/gpqa/gpqa_extended_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:49,485 copying build/lib/lm_eval/tasks/leaderboard/gpqa/_leaderboard_gpqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:49,487 copying build/lib/lm_eval/tasks/leaderboard/gpqa/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:49,489 copying build/lib/lm_eval/tasks/leaderboard/gpqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:49,491 copying build/lib/lm_eval/tasks/leaderboard/gpqa/gpqa_diamond_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T14:29:49,494 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:49,495 copying build/lib/lm_eval/tasks/leaderboard/ifeval/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:49,498 copying build/lib/lm_eval/tasks/leaderboard/ifeval/instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:49,501 copying build/lib/lm_eval/tasks/leaderboard/ifeval/ifeval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:49,503 copying build/lib/lm_eval/tasks/leaderboard/ifeval/_leaderboard_instruction_following.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:49,504 copying build/lib/lm_eval/tasks/leaderboard/ifeval/instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:49,508 copying build/lib/lm_eval/tasks/leaderboard/ifeval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T14:29:49,510 copying build/lib/lm_eval/tasks/leaderboard/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard 2026-05-11T14:29:49,513 copying build/lib/lm_eval/tasks/leaderboard/leaderboard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard 2026-05-11T14:29:49,516 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,517 copying build/lib/lm_eval/tasks/leaderboard/math/_leaderboard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,519 copying build/lib/lm_eval/tasks/leaderboard/math/math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,521 copying build/lib/lm_eval/tasks/leaderboard/math/math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,523 copying build/lib/lm_eval/tasks/leaderboard/math/math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,525 copying build/lib/lm_eval/tasks/leaderboard/math/math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,527 copying build/lib/lm_eval/tasks/leaderboard/math/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,530 copying build/lib/lm_eval/tasks/leaderboard/math/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,533 copying build/lib/lm_eval/tasks/leaderboard/math/math_precalculus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,535 copying build/lib/lm_eval/tasks/leaderboard/math/math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,537 copying build/lib/lm_eval/tasks/leaderboard/math/math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T14:29:49,540 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T14:29:49,541 copying build/lib/lm_eval/tasks/leaderboard/mmlu_pro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T14:29:49,544 copying build/lib/lm_eval/tasks/leaderboard/mmlu_pro/mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T14:29:49,547 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,548 copying build/lib/lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,551 copying build/lib/lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,553 copying build/lib/lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,555 copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,557 copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,560 copying build/lib/lm_eval/tasks/hendrycks_ethics/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,562 copying build/lib/lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,564 copying build/lib/lm_eval/tasks/hendrycks_ethics/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T14:29:49,566 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/squad_completion 2026-05-11T14:29:49,568 copying build/lib/lm_eval/tasks/squad_completion/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squad_completion 2026-05-11T14:29:49,570 copying build/lib/lm_eval/tasks/squad_completion/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squad_completion 2026-05-11T14:29:49,572 copying build/lib/lm_eval/tasks/squad_completion/squad_completion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squad_completion 2026-05-11T14:29:49,574 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa 2026-05-11T14:29:49,575 copying build/lib/lm_eval/tasks/triviaqa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/triviaqa 2026-05-11T14:29:49,578 copying build/lib/lm_eval/tasks/triviaqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/triviaqa 2026-05-11T14:29:49,582 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2026-05-11T14:29:49,584 copying build/lib/lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,586 copying build/lib/lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,588 copying build/lib/lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,590 copying build/lib/lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,592 copying build/lib/lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,594 copying build/lib/lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,596 copying build/lib/lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,598 copying build/lib/lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,600 copying build/lib/lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,602 copying build/lib/lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,604 copying build/lib/lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,606 copying build/lib/lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,608 copying build/lib/lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,610 copying build/lib/lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,611 copying build/lib/lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,613 copying build/lib/lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,615 copying build/lib/lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,616 copying build/lib/lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,618 copying build/lib/lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,620 copying build/lib/lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,621 copying build/lib/lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,623 copying build/lib/lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,624 copying build/lib/lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,626 copying build/lib/lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,628 copying build/lib/lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,629 copying build/lib/lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,631 copying build/lib/lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,633 copying build/lib/lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,635 copying build/lib/lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,636 copying build/lib/lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,638 copying build/lib/lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,640 copying build/lib/lm_eval/tasks/belebele/_belebele.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,642 copying build/lib/lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,644 copying build/lib/lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,645 copying build/lib/lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,647 copying build/lib/lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,649 copying build/lib/lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,650 copying build/lib/lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,652 copying build/lib/lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,654 copying build/lib/lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,656 copying build/lib/lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,657 copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,659 copying build/lib/lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,661 copying build/lib/lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,662 copying build/lib/lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,664 copying build/lib/lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,665 copying build/lib/lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,667 copying build/lib/lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,669 copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,670 copying build/lib/lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,672 copying build/lib/lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,674 copying build/lib/lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,675 copying build/lib/lm_eval/tasks/belebele/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,677 copying build/lib/lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,679 copying build/lib/lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,680 copying build/lib/lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,682 copying build/lib/lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,684 copying build/lib/lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,686 copying build/lib/lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,687 copying build/lib/lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,689 copying build/lib/lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,691 copying build/lib/lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,692 copying build/lib/lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,694 copying build/lib/lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,696 copying build/lib/lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,697 copying build/lib/lm_eval/tasks/belebele/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,699 copying build/lib/lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,701 copying build/lib/lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,703 copying build/lib/lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,705 copying build/lib/lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,706 copying build/lib/lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,708 copying build/lib/lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,709 copying build/lib/lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,711 copying build/lib/lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,712 copying build/lib/lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,714 copying build/lib/lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,716 copying build/lib/lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,717 copying build/lib/lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,719 copying build/lib/lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,721 copying build/lib/lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,722 copying build/lib/lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,724 copying build/lib/lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,726 copying build/lib/lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,728 copying build/lib/lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,730 copying build/lib/lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,732 copying build/lib/lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,733 copying build/lib/lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,735 copying build/lib/lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,737 copying build/lib/lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,738 copying build/lib/lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,740 copying build/lib/lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,742 copying build/lib/lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,744 copying build/lib/lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,745 copying build/lib/lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,747 copying build/lib/lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,749 copying build/lib/lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,751 copying build/lib/lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,753 copying build/lib/lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,754 copying build/lib/lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,756 copying build/lib/lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,757 copying build/lib/lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,759 copying build/lib/lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,761 copying build/lib/lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,762 copying build/lib/lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,764 copying build/lib/lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,766 copying build/lib/lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,767 copying build/lib/lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,769 copying build/lib/lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,771 copying build/lib/lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,772 copying build/lib/lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,774 copying build/lib/lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,776 copying build/lib/lm_eval/tasks/belebele/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,778 copying build/lib/lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,779 copying build/lib/lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,781 copying build/lib/lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,783 copying build/lib/lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,784 copying build/lib/lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,786 copying build/lib/lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,788 copying build/lib/lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,789 copying build/lib/lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,791 copying build/lib/lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,793 copying build/lib/lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,795 copying build/lib/lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,796 copying build/lib/lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,798 copying build/lib/lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,800 copying build/lib/lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T14:29:49,802 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/basque_bench 2026-05-11T14:29:49,803 copying build/lib/lm_eval/tasks/basque_bench/arc_eu_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,805 copying build/lib/lm_eval/tasks/basque_bench/paws_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,806 copying build/lib/lm_eval/tasks/basque_bench/xcopa_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,808 copying build/lib/lm_eval/tasks/basque_bench/arc_eu_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,809 copying build/lib/lm_eval/tasks/basque_bench/mgsm_native_cot_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,811 copying build/lib/lm_eval/tasks/basque_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,814 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,815 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_ca-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,817 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,819 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_pt-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,820 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_de-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,822 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,824 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,825 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_gl-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,827 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,829 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/create_yamls_flores_eu.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,831 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,833 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_fr-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,835 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,837 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_es-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,838 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_en-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,840 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,842 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_it-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,844 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,845 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,847 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T14:29:49,849 copying build/lib/lm_eval/tasks/basque_bench/wnli_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,850 copying build/lib/lm_eval/tasks/basque_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,852 copying build/lib/lm_eval/tasks/basque_bench/piqa_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,854 copying build/lib/lm_eval/tasks/basque_bench/basque_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,855 copying build/lib/lm_eval/tasks/basque_bench/mgsm_direct_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T14:29:49,857 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2026-05-11T14:29:49,858 copying build/lib/lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,860 copying build/lib/lm_eval/tasks/kobest/kobest_copa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,862 copying build/lib/lm_eval/tasks/kobest/_kobest.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,864 copying build/lib/lm_eval/tasks/kobest/kobest_boolq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,865 copying build/lib/lm_eval/tasks/kobest/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,867 copying build/lib/lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,869 copying build/lib/lm_eval/tasks/kobest/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,871 copying build/lib/lm_eval/tasks/kobest/kobest_wic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T14:29:49,873 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,874 copying build/lib/lm_eval/tasks/inverse_scaling/_some_results -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,876 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_repetitive_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,878 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_sig_figs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,880 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_redefine_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,881 copying build/lib/lm_eval/tasks/inverse_scaling/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,883 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_modus_tollens.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,885 copying build/lib/lm_eval/tasks/inverse_scaling/_inverse_scaling_mc_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,887 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_neqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,888 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_pattern_matching_suppression.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,890 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_quote_repetition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,892 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_memo_trap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,893 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_winobias_antistereotype.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,895 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_into_the_unknown.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,897 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_hindsight_neglect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T14:29:49,899 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa 2026-05-11T14:29:49,900 copying build/lib/lm_eval/tasks/logiqa/utils_logiqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa 2026-05-11T14:29:49,902 copying build/lib/lm_eval/tasks/logiqa/logiqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa 2026-05-11T14:29:49,903 copying build/lib/lm_eval/tasks/logiqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa 2026-05-11T14:29:49,905 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual 2026-05-11T14:29:49,906 copying build/lib/lm_eval/tasks/mutual/mutual.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T14:29:49,908 copying build/lib/lm_eval/tasks/mutual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T14:29:49,910 copying build/lib/lm_eval/tasks/mutual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T14:29:49,912 copying build/lib/lm_eval/tasks/mutual/multual_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T14:29:49,914 copying build/lib/lm_eval/tasks/_index.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T14:29:49,917 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/galician_bench 2026-05-11T14:29:49,918 copying build/lib/lm_eval/tasks/galician_bench/xstorycloze_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,921 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,922 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_fr-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,923 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,925 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,927 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,929 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,931 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_it-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,933 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_eu-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,934 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_es-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,936 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,938 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,940 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_en-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,941 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_ca-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,944 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,945 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/create_yamls_flores_gl.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,947 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,949 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,951 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_pt-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,952 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_de-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,954 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T14:29:49,956 copying build/lib/lm_eval/tasks/galician_bench/summarization_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,957 copying build/lib/lm_eval/tasks/galician_bench/openbookqa_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,959 copying build/lib/lm_eval/tasks/galician_bench/paws_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,961 copying build/lib/lm_eval/tasks/galician_bench/parafrases_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,963 copying build/lib/lm_eval/tasks/galician_bench/truthfulqa_gl_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,964 copying build/lib/lm_eval/tasks/galician_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,966 copying build/lib/lm_eval/tasks/galician_bench/truthfulqa_gl_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,968 copying build/lib/lm_eval/tasks/galician_bench/galcola.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,970 copying build/lib/lm_eval/tasks/galician_bench/xnli_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,972 copying build/lib/lm_eval/tasks/galician_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,974 copying build/lib/lm_eval/tasks/galician_bench/mgsm_direct_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,976 copying build/lib/lm_eval/tasks/galician_bench/truthfulqa_gl_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,977 copying build/lib/lm_eval/tasks/galician_bench/belebele_glg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,979 copying build/lib/lm_eval/tasks/galician_bench/galician_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T14:29:49,982 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2026-05-11T14:29:49,983 copying build/lib/lm_eval/tasks/xcopa/default_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,985 copying build/lib/lm_eval/tasks/xcopa/default_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,987 copying build/lib/lm_eval/tasks/xcopa/default_ht.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,989 copying build/lib/lm_eval/tasks/xcopa/default_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,990 copying build/lib/lm_eval/tasks/xcopa/default_qu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,992 copying build/lib/lm_eval/tasks/xcopa/default_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,994 copying build/lib/lm_eval/tasks/xcopa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,995 copying build/lib/lm_eval/tasks/xcopa/_xcopa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,997 copying build/lib/lm_eval/tasks/xcopa/default_et.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:49,999 copying build/lib/lm_eval/tasks/xcopa/default_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:50,001 copying build/lib/lm_eval/tasks/xcopa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:50,003 copying build/lib/lm_eval/tasks/xcopa/default_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:50,004 copying build/lib/lm_eval/tasks/xcopa/default_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:50,006 copying build/lib/lm_eval/tasks/xcopa/default_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T14:29:50,009 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cnn_dailymail 2026-05-11T14:29:50,010 copying build/lib/lm_eval/tasks/cnn_dailymail/cnn_dailymail.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cnn_dailymail 2026-05-11T14:29:50,012 copying build/lib/lm_eval/tasks/cnn_dailymail/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cnn_dailymail 2026-05-11T14:29:50,014 copying build/lib/lm_eval/tasks/cnn_dailymail/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cnn_dailymail 2026-05-11T14:29:50,017 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2026-05-11T14:29:50,018 copying build/lib/lm_eval/tasks/ifeval/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T14:29:50,021 copying build/lib/lm_eval/tasks/ifeval/instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T14:29:50,025 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval/multilingual 2026-05-11T14:29:50,029 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:29:50,031 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions/ca_instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:29:50,034 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions/es_instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:29:50,038 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T14:29:50,040 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T14:29:50,042 copying build/lib/lm_eval/tasks/ifeval/multilingual/ifeval_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T14:29:50,046 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:29:50,047 copying build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils/ca_instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:29:50,050 copying build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:29:50,051 copying build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils/es_instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T14:29:50,053 copying build/lib/lm_eval/tasks/ifeval/multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T14:29:50,055 copying build/lib/lm_eval/tasks/ifeval/multilingual/ifeval_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T14:29:50,057 copying build/lib/lm_eval/tasks/ifeval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T14:29:50,059 copying build/lib/lm_eval/tasks/ifeval/ifeval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T14:29:50,061 copying build/lib/lm_eval/tasks/ifeval/instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T14:29:50,064 copying build/lib/lm_eval/tasks/ifeval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T14:29:50,067 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice 2026-05-11T14:29:50,072 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:50,073 copying build/lib/lm_eval/tasks/aradice/winogrande/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:50,075 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:50,076 copying build/lib/lm_eval/tasks/aradice/winogrande/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:50,078 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:50,080 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:50,082 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T14:29:50,085 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:50,086 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:50,088 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:50,090 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:50,091 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:50,093 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T14:29:50,096 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/ArabicMMLU 2026-05-11T14:29:50,098 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,099 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,101 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,103 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,104 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,106 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,108 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,109 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,111 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,113 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,115 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,116 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,118 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,120 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,121 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,123 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,125 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,127 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,128 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,130 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,132 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,134 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,135 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,137 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,139 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,140 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,142 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,144 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,146 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,147 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,149 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,151 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,152 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,154 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,156 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,158 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,159 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,161 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,163 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,165 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,167 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,168 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,170 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,172 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,174 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T14:29:50,177 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,178 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,180 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,182 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,183 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,185 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,187 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,189 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,190 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,192 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,194 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,196 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,198 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,199 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,201 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,203 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,204 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,206 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,208 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,210 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,212 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,213 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,215 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,217 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,218 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,220 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,222 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,224 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,225 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,227 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,229 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,231 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,232 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,234 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,236 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,238 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,240 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,242 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,244 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,246 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,248 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,249 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,251 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,253 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,255 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T14:29:50,257 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,258 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,260 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,262 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,264 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,266 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,267 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,269 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/qatar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,271 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T14:29:50,273 copying build/lib/lm_eval/tasks/aradice/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice 2026-05-11T14:29:50,275 copying build/lib/lm_eval/tasks/aradice/aradice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice 2026-05-11T14:29:50,277 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/piqa 2026-05-11T14:29:50,278 copying build/lib/lm_eval/tasks/aradice/piqa/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T14:29:50,280 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T14:29:50,282 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T14:29:50,284 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T14:29:50,287 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T14:29:50,289 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:50,290 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:50,292 copying build/lib/lm_eval/tasks/aradice/openbookqa/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:50,294 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:50,296 copying build/lib/lm_eval/tasks/aradice/openbookqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:50,297 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:50,299 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T14:29:50,302 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq 2026-05-11T14:29:50,303 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:29:50,304 copying build/lib/lm_eval/tasks/aradice/boolq/ENG/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:29:50,306 copying build/lib/lm_eval/tasks/aradice/boolq/ENG/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:29:50,308 copying build/lib/lm_eval/tasks/aradice/boolq/ENG/boolq_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/ENG 2026-05-11T14:29:50,310 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:29:50,311 copying build/lib/lm_eval/tasks/aradice/boolq/LEV/boolq_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:29:50,313 copying build/lib/lm_eval/tasks/aradice/boolq/LEV/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:29:50,315 copying build/lib/lm_eval/tasks/aradice/boolq/LEV/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/LEV 2026-05-11T14:29:50,318 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:29:50,320 copying build/lib/lm_eval/tasks/aradice/boolq/MSA/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:29:50,322 copying build/lib/lm_eval/tasks/aradice/boolq/MSA/boolq_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:29:50,323 copying build/lib/lm_eval/tasks/aradice/boolq/MSA/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/MSA 2026-05-11T14:29:50,326 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:29:50,327 copying build/lib/lm_eval/tasks/aradice/boolq/EGY/boolq_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:29:50,329 copying build/lib/lm_eval/tasks/aradice/boolq/EGY/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:29:50,331 copying build/lib/lm_eval/tasks/aradice/boolq/EGY/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/EGY 2026-05-11T14:29:50,333 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,334 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,336 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,338 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,340 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,342 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,344 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,345 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,347 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T14:29:50,349 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:50,350 copying build/lib/lm_eval/tasks/mediqa_qa2019/mediqa_qa2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:50,352 copying build/lib/lm_eval/tasks/mediqa_qa2019/mediqa_qa2019_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:50,354 copying build/lib/lm_eval/tasks/mediqa_qa2019/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:50,356 copying build/lib/lm_eval/tasks/mediqa_qa2019/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:50,358 copying build/lib/lm_eval/tasks/mediqa_qa2019/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T14:29:50,360 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2026-05-11T14:29:50,361 copying build/lib/lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,363 copying build/lib/lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,365 copying build/lib/lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,366 copying build/lib/lm_eval/tasks/csatqa/_csatqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,368 copying build/lib/lm_eval/tasks/csatqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,370 copying build/lib/lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,372 copying build/lib/lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,374 copying build/lib/lm_eval/tasks/csatqa/csatqa_li.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,376 copying build/lib/lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,378 copying build/lib/lm_eval/tasks/csatqa/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T14:29:50,380 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi 2026-05-11T14:29:50,382 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,383 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,385 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,387 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,389 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,391 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,393 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,395 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,397 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,398 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,400 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,402 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,403 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,405 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,407 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,409 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,410 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,412 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,414 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,416 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,418 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,419 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,421 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,423 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,425 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,426 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,428 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,430 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,432 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,434 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,435 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,437 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,439 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,440 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T14:29:50,443 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,444 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,446 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,448 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,450 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/_arc_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,452 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,453 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,455 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,457 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,459 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,460 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,462 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,464 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,466 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,468 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,470 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,471 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,473 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,475 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,477 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,479 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,480 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,482 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,484 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,485 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,487 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,489 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,490 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,492 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,494 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,496 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,497 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,499 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,501 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,503 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T14:29:50,506 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,507 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,509 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,511 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,512 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,514 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,516 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,519 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,521 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,522 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,524 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,526 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,527 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,529 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,531 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,532 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,534 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,536 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,537 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,539 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,541 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,543 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,544 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,546 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,548 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,550 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,551 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,553 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,555 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,556 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,558 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,560 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,562 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,563 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,565 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,567 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,569 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T14:29:50,572 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,573 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,575 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,577 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,579 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,580 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,582 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,584 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,586 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,588 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,590 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,592 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,594 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,596 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,598 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,600 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,602 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,605 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,607 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,608 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,610 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,612 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,613 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,615 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,617 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,619 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,621 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,623 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,625 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,627 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,628 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,630 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,632 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,634 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,635 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,637 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,639 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,641 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,642 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,644 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,646 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,648 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,649 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,651 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,653 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,655 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,656 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,658 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,660 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,662 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,663 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,665 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,667 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,669 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,671 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,672 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,674 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,676 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,678 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,679 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,681 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,683 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,684 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,686 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,688 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,689 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,691 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T14:29:50,693 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/graphwalks 2026-05-11T14:29:50,694 copying build/lib/lm_eval/tasks/graphwalks/graphwalks_128k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T14:29:50,696 copying build/lib/lm_eval/tasks/graphwalks/graphwalks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T14:29:50,698 copying build/lib/lm_eval/tasks/graphwalks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T14:29:50,700 copying build/lib/lm_eval/tasks/graphwalks/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T14:29:50,702 copying build/lib/lm_eval/tasks/graphwalks/graphwalks_1M.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T14:29:50,704 copying build/lib/lm_eval/tasks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T14:29:50,708 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2026-05-11T14:29:50,709 copying build/lib/lm_eval/tasks/fld/fld_logical_formula_default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T14:29:50,711 copying build/lib/lm_eval/tasks/fld/fld_default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T14:29:50,713 copying build/lib/lm_eval/tasks/fld/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T14:29:50,715 copying build/lib/lm_eval/tasks/fld/fld_star.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T14:29:50,717 copying build/lib/lm_eval/tasks/fld/fld_logical_formula_star.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T14:29:50,719 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2026-05-11T14:29:50,720 copying build/lib/lm_eval/tasks/aexams/aexams_Social.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,722 copying build/lib/lm_eval/tasks/aexams/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,724 copying build/lib/lm_eval/tasks/aexams/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,726 copying build/lib/lm_eval/tasks/aexams/aexams_Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,727 copying build/lib/lm_eval/tasks/aexams/aexams_Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,729 copying build/lib/lm_eval/tasks/aexams/aexams_Science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,731 copying build/lib/lm_eval/tasks/aexams/_aexams.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,732 copying build/lib/lm_eval/tasks/aexams/aexams_IslamicStudies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T14:29:50,735 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli 2026-05-11T14:29:50,736 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct 2026-05-11T14:29:50,738 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,739 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,740 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,742 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,744 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,745 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,747 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,749 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,751 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,752 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,754 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,756 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,758 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,760 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,761 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,763 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,765 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,767 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,768 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,770 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,772 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T14:29:50,774 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,775 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,777 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,779 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,780 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,782 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,784 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,785 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,787 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,789 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,790 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,792 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,794 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,795 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,797 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,799 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,801 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,802 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,804 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,806 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,808 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T14:29:50,810 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,811 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,813 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,815 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,816 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,818 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,820 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,822 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,823 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,825 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,827 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,828 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,830 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,832 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,833 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,835 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,837 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,839 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,840 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,842 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,844 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T14:29:50,846 copying build/lib/lm_eval/tasks/afrixnli/direct/afrixnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct 2026-05-11T14:29:50,848 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,849 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,851 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,853 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,854 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,857 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,858 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,860 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,862 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,863 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,865 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,867 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,868 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,870 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,872 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,874 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,875 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,877 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,879 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,881 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,883 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T14:29:50,885 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,886 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,888 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,890 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,891 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,893 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,895 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,897 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,898 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,900 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,902 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,903 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,905 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,907 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,908 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,910 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,912 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,913 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,915 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,917 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,918 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T14:29:50,921 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate 2026-05-11T14:29:50,923 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,924 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,926 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,928 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,930 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,932 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,933 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,935 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,937 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,939 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,941 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,942 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,944 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,946 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,948 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,949 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,951 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,953 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,954 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,956 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T14:29:50,958 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,959 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,961 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,963 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,965 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,967 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,968 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,970 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,972 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,973 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,975 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,977 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,979 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,980 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,982 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,984 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,985 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,987 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,989 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,990 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T14:29:50,993 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:50,994 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:50,996 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:50,998 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:50,999 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,001 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,003 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,005 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,006 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,008 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,010 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,012 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,013 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,015 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,017 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,019 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,020 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,022 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,024 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,025 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T14:29:51,028 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,029 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,031 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,033 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,034 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,036 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,038 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,040 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,041 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,043 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,045 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,047 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,049 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,050 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,052 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,054 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,056 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,058 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,059 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,061 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T14:29:51,063 copying build/lib/lm_eval/tasks/afrixnli/translate/afrixnli_tt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate 2026-05-11T14:29:51,065 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,066 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,068 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,070 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,072 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,073 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,075 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,077 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,079 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,081 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,083 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,084 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,086 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,088 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,089 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,091 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,093 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,095 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,097 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,098 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T14:29:51,100 copying build/lib/lm_eval/tasks/afrixnli/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli 2026-05-11T14:29:51,102 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt 2026-05-11T14:29:51,104 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,105 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,107 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,108 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,110 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,112 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,113 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,115 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,116 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,118 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,120 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,121 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,123 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,125 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,126 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,128 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,130 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,132 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,134 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,135 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,137 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T14:29:51,139 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,140 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,142 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,144 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,146 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,148 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,149 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,151 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,153 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,155 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,157 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,159 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,161 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,163 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,165 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,167 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,169 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,170 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,172 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,174 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T14:29:51,177 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,178 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,180 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,182 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,184 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,186 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,188 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,190 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,192 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,194 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,195 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,197 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,199 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,200 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,202 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,204 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,206 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,208 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,209 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,211 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,212 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T14:29:51,214 copying build/lib/lm_eval/tasks/afrixnli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli 2026-05-11T14:29:51,217 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/lai prompt 2026-05-11T14:29:51,219 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,220 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,221 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,223 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,225 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,226 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,228 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,230 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,232 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,234 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,235 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,237 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,239 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,241 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,243 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,245 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,246 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,248 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,250 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,251 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,253 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T14:29:51,256 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,257 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,259 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,260 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,262 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,264 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,266 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,268 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,269 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,271 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,273 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,275 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,277 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,278 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,280 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,282 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,284 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,286 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,287 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,289 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T14:29:51,291 copying build/lib/lm_eval/tasks/afrixnli/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli 2026-05-11T14:29:51,293 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2026-05-11T14:29:51,294 copying build/lib/lm_eval/tasks/wmdp/wmdp_chem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T14:29:51,296 copying build/lib/lm_eval/tasks/wmdp/wmdp_bio.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T14:29:51,298 copying build/lib/lm_eval/tasks/wmdp/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T14:29:51,300 copying build/lib/lm_eval/tasks/wmdp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T14:29:51,302 copying build/lib/lm_eval/tasks/wmdp/_wmdp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T14:29:51,303 copying build/lib/lm_eval/tasks/wmdp/wmdp_cyber.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T14:29:51,306 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/c4 2026-05-11T14:29:51,307 copying build/lib/lm_eval/tasks/c4/preprocess_c4.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/c4 2026-05-11T14:29:51,309 copying build/lib/lm_eval/tasks/c4/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/c4 2026-05-11T14:29:51,311 copying build/lib/lm_eval/tasks/c4/c4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/c4 2026-05-11T14:29:51,313 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bangla 2026-05-11T14:29:51,314 copying build/lib/lm_eval/tasks/bangla/bangla_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T14:29:51,316 copying build/lib/lm_eval/tasks/bangla/bangla_openbookqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T14:29:51,318 copying build/lib/lm_eval/tasks/bangla/bangla_commonsenseqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T14:29:51,320 copying build/lib/lm_eval/tasks/bangla/bangla_boolqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T14:29:51,322 copying build/lib/lm_eval/tasks/bangla/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T14:29:51,324 copying build/lib/lm_eval/tasks/bangla/bangla_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T14:29:51,326 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite 2026-05-11T14:29:51,328 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,329 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_spelled_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,331 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_equivalent_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,333 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_base.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,335 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_canonical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,337 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_traditional.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,339 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_romanization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,340 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_colloquial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,342 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_keyboard_proximity_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,344 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/_toksuite_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,345 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_ocr_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,347 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_code_language_script_switching.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,349 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_word_spacing_zero-width_characters_extra_space.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,351 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_space_removal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,352 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_partially_romanized.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,354 copying build/lib/lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_optional_diacritics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_chinese 2026-05-11T14:29:51,356 copying build/lib/lm_eval/tasks/toksuite/_toksuite_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite 2026-05-11T14:29:51,359 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,360 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_colloquial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,362 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_base.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,363 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_keyboard_proximity_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,365 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_dialects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,367 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_number_romanization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,369 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_arabic_keyboard_for_farsi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,371 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_equivalent_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,372 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_spelled_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,374 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_code_language_script_switching.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,376 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_word_spacing_zero-width_characters_extra_space.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,378 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/_toksuite_farsi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,379 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_canonical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,381 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_romanization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,383 copying build/lib/lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_optional_diacritics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_farsi 2026-05-11T14:29:51,385 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,386 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_latex.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,388 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,390 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_spelled_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,391 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_farsi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,393 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,395 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_decorative_unicode.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,396 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,398 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_base.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,400 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/_toksuite_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,401 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_space_removal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,403 copying build/lib/lm_eval/tasks/toksuite/toksuite_math/toksuite_math_canonical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_math 2026-05-11T14:29:51,405 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,406 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_plausible_diacritics_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,408 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_orthographic_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,410 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_code_language_script_switching.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,412 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_keyboard_proximity_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,413 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_base.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,415 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_similar_words.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,417 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_web_search_query.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,419 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_date_formats.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,420 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_contractions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,422 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_grammatical_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,424 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/_toksuite_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,426 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_numerical_formats.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,428 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_typographical_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,429 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_dialects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,431 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_english_keyboard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,433 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_phonetic_spelling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,435 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_capitalization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,437 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_spelled_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,438 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_canonical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,440 copying build/lib/lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_abbreviations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_italian 2026-05-11T14:29:51,442 copying build/lib/lm_eval/tasks/toksuite/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite 2026-05-11T14:29:51,445 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,447 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_morpheme_separation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,449 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_typographical_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,451 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_unicode_formatting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,453 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_strikethrough.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,455 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_colloquial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,457 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_fullwidth_characters.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,458 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_character_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,460 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_scripted_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,462 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_spelled_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,464 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_enclosed_characters.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,466 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_double_struck.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,468 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_upside_down_rotated.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,470 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_base.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,472 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/_toksuite_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,473 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_unusual_formatting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,475 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_diacriticized_styling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,477 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_superscript_subscript.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,479 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_space_removal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,480 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_equivalent_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,482 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_canonical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,484 copying build/lib/lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_latex.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_stem 2026-05-11T14:29:51,487 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,488 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_spelled_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,490 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_similar_words.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,491 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/_toksuite_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,493 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_dialects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,495 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_grammatical_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,497 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_canonical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,499 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_derivations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,501 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_keyboard_proximity_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,502 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_code_language_script_switching.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,504 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_base.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,506 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_inflections.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,508 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_english_keyboard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,509 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_colloquial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,511 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_word_reordering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,513 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_equivalent_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,515 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_date_formats.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,517 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_typographical_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,519 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_web_search_query.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,521 copying build/lib/lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_orthographic_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_turkish 2026-05-11T14:29:51,523 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,524 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_emoji_substitution.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,527 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_date_formats.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,528 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_spaced_styling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,530 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_homoglyphs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,532 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_word_reordering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,534 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_space_removal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,536 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_letter_repetition_for_emphasis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,537 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_contractions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,539 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_lowercase.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,541 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_abbreviations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,543 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_compounds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,545 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_inflections.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,546 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_colloquial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,548 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_ocr_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,550 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_scripted_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,552 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_superscript_subscript_styling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,553 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_hyphenated_spelling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,555 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_base.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,557 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_orthographic_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,559 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_character_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,561 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_character_substitution.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,563 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_web_search_query.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,564 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_historical_spelling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,567 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/_toksuite_english.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,568 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_capitalization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,570 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_keyboard_proximity_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,572 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_canonical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,574 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_spelled_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,576 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_similar_words.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,577 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_grammatical_errors.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,579 copying build/lib/lm_eval/tasks/toksuite/toksuite_english/toksuite_english_macron_diacritic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite/toksuite_english 2026-05-11T14:29:51,581 copying build/lib/lm_eval/tasks/toksuite/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite 2026-05-11T14:29:51,584 copying build/lib/lm_eval/tasks/toksuite/_toksuite_abridged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toksuite 2026-05-11T14:29:51,587 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/jfinqa 2026-05-11T14:29:51,588 copying build/lib/lm_eval/tasks/jfinqa/jfinqa_numerical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jfinqa 2026-05-11T14:29:51,590 copying build/lib/lm_eval/tasks/jfinqa/jfinqa_consistency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jfinqa 2026-05-11T14:29:51,592 copying build/lib/lm_eval/tasks/jfinqa/jfinqa_temporal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jfinqa 2026-05-11T14:29:51,594 copying build/lib/lm_eval/tasks/jfinqa/_jfinqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jfinqa 2026-05-11T14:29:51,596 copying build/lib/lm_eval/tasks/jfinqa/test_jfinqa_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jfinqa 2026-05-11T14:29:51,598 copying build/lib/lm_eval/tasks/jfinqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jfinqa 2026-05-11T14:29:51,600 copying build/lib/lm_eval/tasks/jfinqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jfinqa 2026-05-11T14:29:51,603 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:51,604 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:51,606 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-self-consistency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:51,608 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:51,610 copying build/lib/lm_eval/tasks/gsm8k_platinum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:51,612 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:51,613 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T14:29:51,616 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa 2026-05-11T14:29:51,617 copying build/lib/lm_eval/tasks/piqa/piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/piqa 2026-05-11T14:29:51,621 copying build/lib/lm_eval/tasks/piqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/piqa 2026-05-11T14:29:51,623 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016 2026-05-11T14:29:51,624 copying build/lib/lm_eval/tasks/wmt2016/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmt2016 2026-05-11T14:29:51,626 copying build/lib/lm_eval/tasks/wmt2016/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmt2016 2026-05-11T14:29:51,628 copying build/lib/lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmt2016 2026-05-11T14:29:51,631 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mela 2026-05-11T14:29:51,632 copying build/lib/lm_eval/tasks/mela/mela_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,633 copying build/lib/lm_eval/tasks/mela/mela_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,635 copying build/lib/lm_eval/tasks/mela/mela_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,637 copying build/lib/lm_eval/tasks/mela/mela_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,638 copying build/lib/lm_eval/tasks/mela/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,640 copying build/lib/lm_eval/tasks/mela/mela_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,642 copying build/lib/lm_eval/tasks/mela/mela_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,644 copying build/lib/lm_eval/tasks/mela/mela_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,645 copying build/lib/lm_eval/tasks/mela/mela_is.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,647 copying build/lib/lm_eval/tasks/mela/_mela.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,649 copying build/lib/lm_eval/tasks/mela/mela_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,650 copying build/lib/lm_eval/tasks/mela/mela_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T14:29:51,653 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ulqa 2026-05-11T14:29:51,654 copying build/lib/lm_eval/tasks/ulqa/lambada_uyghur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T14:29:51,656 copying build/lib/lm_eval/tasks/ulqa/ulqa_.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T14:29:51,657 copying build/lib/lm_eval/tasks/ulqa/celep1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T14:29:51,659 copying build/lib/lm_eval/tasks/ulqa/celep2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T14:29:51,661 copying build/lib/lm_eval/tasks/ulqa/uleval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T14:29:51,663 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:51,664 copying build/lib/lm_eval/tasks/ulqa/ulut/wsm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:51,666 copying build/lib/lm_eval/tasks/ulqa/ulut/nug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:51,668 copying build/lib/lm_eval/tasks/ulqa/ulut/wub.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:51,669 copying build/lib/lm_eval/tasks/ulqa/ulut/ulut.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:51,671 copying build/lib/lm_eval/tasks/ulqa/ulut/wum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:51,673 copying build/lib/lm_eval/tasks/ulqa/ulut/wag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T14:29:51,675 copying build/lib/lm_eval/tasks/ulqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T14:29:51,676 copying build/lib/lm_eval/tasks/ulqa/ulqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T14:29:51,679 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2026-05-11T14:29:51,680 copying build/lib/lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,682 copying build/lib/lm_eval/tasks/pile/pile_stackexchange.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,683 copying build/lib/lm_eval/tasks/pile/pile_gutenberg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,685 copying build/lib/lm_eval/tasks/pile/pile_pile-cc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,686 copying build/lib/lm_eval/tasks/pile/pile_europarl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,688 copying build/lib/lm_eval/tasks/pile/pile_books3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,690 copying build/lib/lm_eval/tasks/pile/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,691 copying build/lib/lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,693 copying build/lib/lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,694 copying build/lib/lm_eval/tasks/pile/pile_hackernews.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,696 copying build/lib/lm_eval/tasks/pile/pile_uspto.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,698 copying build/lib/lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,699 copying build/lib/lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,701 copying build/lib/lm_eval/tasks/pile/pile_github.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,703 copying build/lib/lm_eval/tasks/pile/pile_philpapers.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,704 copying build/lib/lm_eval/tasks/pile/pile_wikipedia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,706 copying build/lib/lm_eval/tasks/pile/pile_enron.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,708 copying build/lib/lm_eval/tasks/pile/pile_freelaw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,710 copying build/lib/lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,712 copying build/lib/lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,713 copying build/lib/lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,715 copying build/lib/lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,717 copying build/lib/lm_eval/tasks/pile/pile_arxiv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T14:29:51,720 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2026-05-11T14:29:51,721 copying build/lib/lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,723 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,724 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,726 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,728 copying build/lib/lm_eval/tasks/ceval/_default_ceval_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,730 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,732 copying build/lib/lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,734 copying build/lib/lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,736 copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,738 copying build/lib/lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,740 copying build/lib/lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,741 copying build/lib/lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,743 copying build/lib/lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,745 copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,747 copying build/lib/lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,749 copying build/lib/lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,750 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,752 copying build/lib/lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,754 copying build/lib/lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,756 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,758 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,760 copying build/lib/lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,762 copying build/lib/lm_eval/tasks/ceval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,764 copying build/lib/lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,766 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,768 copying build/lib/lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,769 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,771 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,773 copying build/lib/lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,775 copying build/lib/lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,776 copying build/lib/lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,778 copying build/lib/lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,780 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,782 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,783 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,785 copying build/lib/lm_eval/tasks/ceval/_ceval-valid.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,787 copying build/lib/lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,788 copying build/lib/lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,790 copying build/lib/lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,792 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,793 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,795 copying build/lib/lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,797 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,798 copying build/lib/lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,800 copying build/lib/lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,802 copying build/lib/lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,803 copying build/lib/lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,805 copying build/lib/lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,807 copying build/lib/lm_eval/tasks/ceval/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,809 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,811 copying build/lib/lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,813 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,814 copying build/lib/lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,816 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,818 copying build/lib/lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,820 copying build/lib/lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T14:29:51,822 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2 2026-05-11T14:29:51,823 copying build/lib/lm_eval/tasks/polemo2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/polemo2 2026-05-11T14:29:51,825 copying build/lib/lm_eval/tasks/polemo2/polemo2_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/polemo2 2026-05-11T14:29:51,827 copying build/lib/lm_eval/tasks/polemo2/polemo2_in.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/polemo2 2026-05-11T14:29:51,829 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa 2026-05-11T14:29:51,830 copying build/lib/lm_eval/tasks/coqa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/coqa 2026-05-11T14:29:51,832 copying build/lib/lm_eval/tasks/coqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/coqa 2026-05-11T14:29:51,833 copying build/lib/lm_eval/tasks/coqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/coqa 2026-05-11T14:29:51,836 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/olaph 2026-05-11T14:29:51,836 copying build/lib/lm_eval/tasks/olaph/olaph_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T14:29:51,838 copying build/lib/lm_eval/tasks/olaph/olaph.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T14:29:51,840 copying build/lib/lm_eval/tasks/olaph/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T14:29:51,841 copying build/lib/lm_eval/tasks/olaph/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T14:29:51,843 copying build/lib/lm_eval/tasks/olaph/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T14:29:51,845 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/click 2026-05-11T14:29:51,846 copying build/lib/lm_eval/tasks/click/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click 2026-05-11T14:29:51,848 copying build/lib/lm_eval/tasks/click/click.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click 2026-05-11T14:29:51,850 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/click/click_lang 2026-05-11T14:29:51,851 copying build/lib/lm_eval/tasks/click/click_lang/click_lang_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T14:29:51,853 copying build/lib/lm_eval/tasks/click/click_lang/click_lang_function.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T14:29:51,855 copying build/lib/lm_eval/tasks/click/click_lang/_click_lang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T14:29:51,857 copying build/lib/lm_eval/tasks/click/click_lang/click_lang_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T14:29:51,859 copying build/lib/lm_eval/tasks/click/click_lang/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T14:29:51,861 copying build/lib/lm_eval/tasks/click/click_lang/_default_click_lang_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T14:29:51,863 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,864 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,866 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_kpop.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,868 copying build/lib/lm_eval/tasks/click/click_cul/_click_cul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,870 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,871 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,873 copying build/lib/lm_eval/tasks/click/click_cul/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,875 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,878 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_economy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,880 copying build/lib/lm_eval/tasks/click/click_cul/_default_click_cul_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,882 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_society.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,884 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_tradition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T14:29:51,887 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu 2026-05-11T14:29:51,890 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct 2026-05-11T14:29:51,892 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,893 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,895 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,897 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,899 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,902 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,904 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,906 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,908 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,910 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,912 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,914 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,916 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,918 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,920 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,923 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,925 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,927 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,929 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,931 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,933 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T14:29:51,936 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,938 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,940 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,942 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,944 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,946 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,948 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,950 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,952 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,954 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,956 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,959 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,961 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,964 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,966 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,968 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,970 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,973 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,975 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,977 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,979 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T14:29:51,981 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,982 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,984 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,986 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,988 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,989 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,991 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,992 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,994 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,996 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,998 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:51,999 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,001 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,003 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,005 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,006 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,008 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,010 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,012 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,013 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,015 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T14:29:52,017 copying build/lib/lm_eval/tasks/afrimmlu/direct/afrimmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct 2026-05-11T14:29:52,019 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,020 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,022 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,024 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,026 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,028 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,029 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,031 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,033 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,035 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,037 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,038 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,040 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,042 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,044 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,046 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,048 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,050 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,051 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,053 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,055 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T14:29:52,057 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,058 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,060 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,062 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,064 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,066 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,068 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,069 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,071 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,072 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,074 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,076 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,078 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,079 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,081 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,083 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,085 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,086 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,088 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,090 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,092 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T14:29:52,093 copying build/lib/lm_eval/tasks/afrimmlu/fewshot.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T14:29:52,096 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate 2026-05-11T14:29:52,097 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,098 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,100 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,102 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,104 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,106 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,107 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,109 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,111 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,113 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,114 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,116 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,118 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,119 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,121 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,123 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,124 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,126 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,128 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,130 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T14:29:52,132 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,133 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,135 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,137 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,139 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,140 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,142 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,144 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,146 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,147 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,149 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,151 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,152 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,154 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,156 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,157 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,159 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,161 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,163 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,164 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T14:29:52,167 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,168 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,170 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,172 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,173 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,175 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,177 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,178 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,180 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,184 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,185 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,187 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,189 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,190 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,192 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,194 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,196 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,197 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,199 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,201 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T14:29:52,202 copying build/lib/lm_eval/tasks/afrimmlu/translate/afrimmlu_tt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate 2026-05-11T14:29:52,205 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,206 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,208 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,210 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,211 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,213 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,215 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,217 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,219 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,220 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,222 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,224 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,225 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,227 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,229 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,230 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,232 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,234 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,236 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,237 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T14:29:52,240 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,241 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,243 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,245 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,247 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,249 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,251 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,252 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,254 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,256 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,258 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,260 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,262 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,264 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,266 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,268 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,270 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,272 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,274 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,276 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T14:29:52,278 copying build/lib/lm_eval/tasks/afrimmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T14:29:52,280 copying build/lib/lm_eval/tasks/afrimmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T14:29:52,282 copying build/lib/lm_eval/tasks/afrimmlu/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T14:29:52,285 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2026-05-11T14:29:52,286 copying build/lib/lm_eval/tasks/unscramble/random_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T14:29:52,288 copying build/lib/lm_eval/tasks/unscramble/anagrams1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T14:29:52,290 copying build/lib/lm_eval/tasks/unscramble/reversed_words.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T14:29:52,292 copying build/lib/lm_eval/tasks/unscramble/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T14:29:52,295 copying build/lib/lm_eval/tasks/unscramble/anagrams2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T14:29:52,296 copying build/lib/lm_eval/tasks/unscramble/cycle_letters.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T14:29:52,299 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mbpp 2026-05-11T14:29:52,300 copying build/lib/lm_eval/tasks/mbpp/mbpp_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T14:29:52,303 copying build/lib/lm_eval/tasks/mbpp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T14:29:52,305 copying build/lib/lm_eval/tasks/mbpp/mbpp_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T14:29:52,307 copying build/lib/lm_eval/tasks/mbpp/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T14:29:52,309 copying build/lib/lm_eval/tasks/mbpp/mbpp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T14:29:52,312 copying build/lib/lm_eval/tasks/mbpp/mbpp_plus_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T14:29:52,314 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile_10k 2026-05-11T14:29:52,316 copying build/lib/lm_eval/tasks/pile_10k/pile_10k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile_10k 2026-05-11T14:29:52,318 copying build/lib/lm_eval/tasks/pile_10k/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile_10k 2026-05-11T14:29:52,321 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open 2026-05-11T14:29:52,323 copying build/lib/lm_eval/tasks/nq_open/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nq_open 2026-05-11T14:29:52,325 copying build/lib/lm_eval/tasks/nq_open/nq_open.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nq_open 2026-05-11T14:29:52,328 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,330 copying build/lib/lm_eval/tasks/catalan_bench/xnli_va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,332 copying build/lib/lm_eval/tasks/catalan_bench/_cabreu_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,335 copying build/lib/lm_eval/tasks/catalan_bench/xstorycloze_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,337 copying build/lib/lm_eval/tasks/catalan_bench/paws_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,339 copying build/lib/lm_eval/tasks/catalan_bench/cabreu_abstractive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,341 copying build/lib/lm_eval/tasks/catalan_bench/xquad_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,343 copying build/lib/lm_eval/tasks/catalan_bench/catalan_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,345 copying build/lib/lm_eval/tasks/catalan_bench/wnli_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,347 copying build/lib/lm_eval/tasks/catalan_bench/siqa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,349 copying build/lib/lm_eval/tasks/catalan_bench/_arc_ca_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,351 copying build/lib/lm_eval/tasks/catalan_bench/arc_ca_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,353 copying build/lib/lm_eval/tasks/catalan_bench/cocoteros_va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,356 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,357 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,359 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,362 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_eu-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,364 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_de-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,366 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_es-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,368 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,370 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,372 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,374 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/create_yamls_flores_ca.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,376 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_it-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,379 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,380 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,382 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,384 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_pt-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,385 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_fr-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,387 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_en-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,389 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,391 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,393 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_gl-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T14:29:52,395 copying build/lib/lm_eval/tasks/catalan_bench/coqcat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,397 copying build/lib/lm_eval/tasks/catalan_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,399 copying build/lib/lm_eval/tasks/catalan_bench/catalanqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,401 copying build/lib/lm_eval/tasks/catalan_bench/cabreu_extreme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,402 copying build/lib/lm_eval/tasks/catalan_bench/openbookqa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,404 copying build/lib/lm_eval/tasks/catalan_bench/copa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,406 copying build/lib/lm_eval/tasks/catalan_bench/xnli_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,408 copying build/lib/lm_eval/tasks/catalan_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,410 copying build/lib/lm_eval/tasks/catalan_bench/arc_ca_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,412 copying build/lib/lm_eval/tasks/catalan_bench/teca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,413 copying build/lib/lm_eval/tasks/catalan_bench/catcola.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,416 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:52,417 copying build/lib/lm_eval/tasks/catalan_bench/phrases_va/phrases_ca-va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:52,419 copying build/lib/lm_eval/tasks/catalan_bench/phrases_va/phrases_va-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:52,421 copying build/lib/lm_eval/tasks/catalan_bench/phrases_va/_phrases_va_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T14:29:52,423 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T14:29:52,424 copying build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va/truthfulqa_va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T14:29:52,426 copying build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T14:29:52,429 copying build/lib/lm_eval/tasks/catalan_bench/parafraseja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,430 copying build/lib/lm_eval/tasks/catalan_bench/piqa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,432 copying build/lib/lm_eval/tasks/catalan_bench/cabreu_extractive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,434 copying build/lib/lm_eval/tasks/catalan_bench/mgsm_direct_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T14:29:52,435 copying build/lib/lm_eval/tasks/_factory.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T14:29:52,438 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbq 2026-05-11T14:29:52,439 copying build/lib/lm_eval/tasks/bbq/bbq_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,441 copying build/lib/lm_eval/tasks/bbq/bbq_multiple_choice_disambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,442 copying build/lib/lm_eval/tasks/bbq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,444 copying build/lib/lm_eval/tasks/bbq/bbq_generate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,446 copying build/lib/lm_eval/tasks/bbq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,449 copying build/lib/lm_eval/tasks/bbq/bbq_generate_disambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,450 copying build/lib/lm_eval/tasks/bbq/bbq_generate_ambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,452 copying build/lib/lm_eval/tasks/bbq/bbq_multiple_choice_ambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T14:29:52,454 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,455 copying build/lib/lm_eval/tasks/humaneval_infilling/single_line_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,457 copying build/lib/lm_eval/tasks/humaneval_infilling/humaneval_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,459 copying build/lib/lm_eval/tasks/humaneval_infilling/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,461 copying build/lib/lm_eval/tasks/humaneval_infilling/random_span_infilling_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,462 copying build/lib/lm_eval/tasks/humaneval_infilling/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,464 copying build/lib/lm_eval/tasks/humaneval_infilling/multi_line_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,466 copying build/lib/lm_eval/tasks/humaneval_infilling/random_span_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T14:29:52,468 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco 2026-05-11T14:29:52,469 copying build/lib/lm_eval/tasks/mc_taco/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mc_taco 2026-05-11T14:29:52,471 copying build/lib/lm_eval/tasks/mc_taco/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mc_taco 2026-05-11T14:29:52,474 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm 2026-05-11T14:29:52,475 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,477 copying build/lib/lm_eval/tasks/mgsm/direct/direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,479 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,480 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,482 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,484 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,485 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,487 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,489 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,490 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,492 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,494 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,495 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T14:29:52,498 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,499 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,501 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,503 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,505 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,507 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,509 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,512 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,514 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,516 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,518 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,520 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,521 copying build/lib/lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T14:29:52,524 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,525 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,527 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,529 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,531 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,533 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,534 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,536 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,538 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,539 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,541 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,543 copying build/lib/lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,545 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T14:29:52,547 copying build/lib/lm_eval/tasks/mgsm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm 2026-05-11T14:29:52,549 copying build/lib/lm_eval/tasks/mgsm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm 2026-05-11T14:29:52,551 copying build/lib/lm_eval/tasks/mgsm/gen_yaml.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm 2026-05-11T14:29:52,553 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue 2026-05-11T14:29:52,555 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc 2026-05-11T14:29:52,556 copying build/lib/lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/multirc 2026-05-11T14:29:52,558 copying build/lib/lm_eval/tasks/super_glue/multirc/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/multirc 2026-05-11T14:29:52,560 copying build/lib/lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/multirc 2026-05-11T14:29:52,562 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte 2026-05-11T14:29:52,563 copying build/lib/lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/rte 2026-05-11T14:29:52,565 copying build/lib/lm_eval/tasks/super_glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/rte 2026-05-11T14:29:52,568 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc 2026-05-11T14:29:52,569 copying build/lib/lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T14:29:52,571 copying build/lib/lm_eval/tasks/super_glue/wsc/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T14:29:52,572 copying build/lib/lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T14:29:52,574 copying build/lib/lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T14:29:52,576 copying build/lib/lm_eval/tasks/super_glue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue 2026-05-11T14:29:52,579 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb 2026-05-11T14:29:52,580 copying build/lib/lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T14:29:52,581 copying build/lib/lm_eval/tasks/super_glue/cb/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T14:29:52,583 copying build/lib/lm_eval/tasks/super_glue/cb/aggregate.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T14:29:52,585 copying build/lib/lm_eval/tasks/super_glue/cb/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T14:29:52,587 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa 2026-05-11T14:29:52,588 copying build/lib/lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/copa 2026-05-11T14:29:52,590 copying build/lib/lm_eval/tasks/super_glue/copa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/copa 2026-05-11T14:29:52,591 copying build/lib/lm_eval/tasks/super_glue/copa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/copa 2026-05-11T14:29:52,594 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic 2026-05-11T14:29:52,595 copying build/lib/lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wic 2026-05-11T14:29:52,597 copying build/lib/lm_eval/tasks/super_glue/wic/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wic 2026-05-11T14:29:52,599 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:52,600 copying build/lib/lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:52,602 copying build/lib/lm_eval/tasks/super_glue/boolq/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:52,604 copying build/lib/lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/boolq 2026-05-11T14:29:52,606 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record 2026-05-11T14:29:52,607 copying build/lib/lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T14:29:52,609 copying build/lib/lm_eval/tasks/super_glue/record/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T14:29:52,611 copying build/lib/lm_eval/tasks/super_glue/record/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T14:29:52,613 copying build/lib/lm_eval/tasks/super_glue/record/util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T14:29:52,616 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa 2026-05-11T14:29:52,617 copying build/lib/lm_eval/tasks/medqa/medqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medqa 2026-05-11T14:29:52,618 copying build/lib/lm_eval/tasks/medqa/preprocess_medqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medqa 2026-05-11T14:29:52,621 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mimic_repsum 2026-05-11T14:29:52,622 copying build/lib/lm_eval/tasks/mimic_repsum/mimic_repsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T14:29:52,624 copying build/lib/lm_eval/tasks/mimic_repsum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T14:29:52,625 copying build/lib/lm_eval/tasks/mimic_repsum/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T14:29:52,627 copying build/lib/lm_eval/tasks/mimic_repsum/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T14:29:52,629 copying build/lib/lm_eval/tasks/mimic_repsum/mimic_repsum_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T14:29:52,632 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2026-05-11T14:29:52,633 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,635 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,636 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,638 copying build/lib/lm_eval/tasks/xwinograd/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,640 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,641 copying build/lib/lm_eval/tasks/xwinograd/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,643 copying build/lib/lm_eval/tasks/xwinograd/_xwinograd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,645 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,646 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,648 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T14:29:52,650 copying build/lib/lm_eval/tasks/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T14:29:52,653 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2026-05-11T14:29:52,654 copying build/lib/lm_eval/tasks/qasper/freeform.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T14:29:52,656 copying build/lib/lm_eval/tasks/qasper/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T14:29:52,657 copying build/lib/lm_eval/tasks/qasper/bool.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T14:29:52,659 copying build/lib/lm_eval/tasks/qasper/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T14:29:52,661 copying build/lib/lm_eval/tasks/qasper/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T14:29:52,663 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus 2026-05-11T14:29:52,664 copying build/lib/lm_eval/tasks/tmmluplus/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus 2026-05-11T14:29:52,667 copying build/lib/lm_eval/tasks/tmmluplus/subject.tsv -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus 2026-05-11T14:29:52,670 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,671 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,673 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,675 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,677 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,679 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,680 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,682 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,684 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,686 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,687 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,689 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,691 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,692 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,694 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,696 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,698 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,700 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,702 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,703 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,705 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,707 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,709 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,710 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,712 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,714 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,715 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,717 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,719 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_STEM.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,721 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,722 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,724 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,726 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,728 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,729 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,731 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,733 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,734 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,736 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,738 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,739 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,741 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,743 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,745 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,746 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,748 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,750 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,752 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,753 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,755 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,757 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,759 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,761 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,762 copying build/lib/lm_eval/tasks/tmmluplus/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,764 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,766 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,768 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,770 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,771 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,773 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,774 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,776 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,778 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,780 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,781 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,783 copying build/lib/lm_eval/tasks/tmmluplus/default/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,785 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,787 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,789 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,795 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,797 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,799 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,801 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,803 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,805 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,806 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T14:29:52,809 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turkishmmlu 2026-05-11T14:29:52,810 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,811 copying build/lib/lm_eval/tasks/turkishmmlu/config/Turkish_Language_and_Literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,814 copying build/lib/lm_eval/tasks/turkishmmlu/config/_turkishmmlu_default_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,815 copying build/lib/lm_eval/tasks/turkishmmlu/config/Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,817 copying build/lib/lm_eval/tasks/turkishmmlu/config/Religion_and_Ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,819 copying build/lib/lm_eval/tasks/turkishmmlu/config/Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,820 copying build/lib/lm_eval/tasks/turkishmmlu/config/Mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,822 copying build/lib/lm_eval/tasks/turkishmmlu/config/Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,823 copying build/lib/lm_eval/tasks/turkishmmlu/config/History.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,825 copying build/lib/lm_eval/tasks/turkishmmlu/config/Philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,827 copying build/lib/lm_eval/tasks/turkishmmlu/config/Geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T14:29:52,829 copying build/lib/lm_eval/tasks/turkishmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu 2026-05-11T14:29:52,832 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,833 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Turkish_Language_and_Literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,835 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/_turkishmmlu_cot_default_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,837 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,839 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Religion_and_Ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,842 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,844 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,846 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,848 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/History.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,851 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,853 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T14:29:52,856 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm 2026-05-11T14:29:52,857 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct 2026-05-11T14:29:52,859 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,860 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,863 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,864 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,866 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,868 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,869 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,871 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,873 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,874 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,876 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,878 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,879 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,881 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,883 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,884 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,886 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,888 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,890 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,891 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,893 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T14:29:52,896 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,897 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,899 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,901 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,902 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,904 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,906 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,908 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,910 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,911 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,913 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,915 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,916 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,918 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,920 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,922 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,923 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,925 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,927 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,929 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,931 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T14:29:52,933 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,934 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,936 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,938 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,939 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,941 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,943 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,945 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,947 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,949 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,950 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,952 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,954 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,956 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,958 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,960 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,961 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,963 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,965 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,966 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,968 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T14:29:52,970 copying build/lib/lm_eval/tasks/afrimgsm/direct/afrimgsm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct 2026-05-11T14:29:52,972 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,973 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,975 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,977 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,979 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,981 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,982 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,984 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,986 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,988 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,990 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,992 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,993 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,995 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,996 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:52,998 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:53,000 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:53,001 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:53,003 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:53,005 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:53,006 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T14:29:53,009 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,010 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,012 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,013 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,015 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,017 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,018 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,020 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,022 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,024 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,025 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,027 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,029 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,031 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,033 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,034 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,036 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,038 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,039 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,041 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,043 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T14:29:53,045 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate 2026-05-11T14:29:53,046 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,048 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,049 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,051 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,053 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,055 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,057 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,058 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,060 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,062 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,064 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,065 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,067 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,069 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,071 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,073 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,074 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,076 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,078 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T14:29:53,080 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,081 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,083 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,085 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,087 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,088 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,090 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,092 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,094 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,095 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,097 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,099 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,101 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,102 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,104 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,106 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,108 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,109 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,111 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T14:29:53,113 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,114 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,116 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,118 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,120 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,121 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,123 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,125 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,126 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,128 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,130 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,132 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,133 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,135 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,137 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,139 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,140 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,142 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,144 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T14:29:53,146 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,147 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,149 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,151 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,152 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,154 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,156 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,157 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,159 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,161 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,162 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,164 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,166 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,167 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,169 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,171 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,173 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,174 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,176 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T14:29:53,178 copying build/lib/lm_eval/tasks/afrimgsm/translate/afrimgsm_tt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate 2026-05-11T14:29:53,180 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,181 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,183 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,185 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,187 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,188 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,190 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,192 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,193 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,195 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,196 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,198 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,200 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,202 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,203 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,205 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,207 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,209 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,211 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T14:29:53,213 copying build/lib/lm_eval/tasks/afrimgsm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T14:29:53,215 copying build/lib/lm_eval/tasks/afrimgsm/run.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T14:29:53,216 copying build/lib/lm_eval/tasks/afrimgsm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T14:29:53,219 copying build/lib/lm_eval/tasks/afrimgsm/gen_yaml.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T14:29:53,221 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T14:29:53,223 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,224 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,226 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,227 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,229 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,231 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,232 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,234 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,235 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,237 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,239 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,240 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,242 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,244 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,245 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,247 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,249 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,251 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,253 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,254 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,256 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T14:29:53,258 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,259 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,261 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,263 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,265 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,267 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,268 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,270 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,272 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,273 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,275 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,277 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,278 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,280 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,282 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,283 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,285 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,287 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,289 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,291 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,292 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T14:29:53,295 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,296 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,298 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,300 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,301 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,303 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,305 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,307 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,308 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,310 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,312 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,313 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,315 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,317 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,318 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,320 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,322 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,323 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,325 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,327 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,329 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T14:29:53,330 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/afrimgsm_cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T14:29:53,333 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,334 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,335 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,337 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,339 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,341 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,342 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,344 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,346 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,347 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,349 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,351 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,352 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,354 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,356 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,358 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,359 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,361 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,363 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,365 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,367 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T14:29:53,369 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,370 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,372 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,374 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,375 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,377 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,379 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,381 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,382 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,384 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,386 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,388 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,389 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,391 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,393 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,394 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,396 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,398 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,400 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,402 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,403 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T14:29:53,406 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T14:29:53,407 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,408 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,410 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,412 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,414 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,416 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,418 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,419 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,421 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,423 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,424 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,426 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,428 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,429 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,431 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,433 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,434 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,436 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,438 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,439 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T14:29:53,442 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,443 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,445 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,446 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,448 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,450 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,452 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,453 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,455 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,457 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,459 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,460 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,462 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,464 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,465 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,467 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,469 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,471 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,473 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,474 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T14:29:53,477 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/afrimgsm_tt_cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T14:29:53,479 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,480 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,482 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,484 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,486 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,488 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,490 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,492 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,493 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,495 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,497 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,499 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,500 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,502 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,504 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,505 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,507 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,509 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,511 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,513 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T14:29:53,516 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,517 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,520 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,522 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,524 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,526 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,528 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,530 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,532 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,534 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,536 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,538 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,540 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,541 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,543 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,545 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,547 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,549 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,550 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,552 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T14:29:53,555 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,556 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,558 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,560 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,562 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,563 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,565 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,567 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,569 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,571 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,573 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,574 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,576 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,578 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,579 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,581 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,583 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,585 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,587 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,589 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T14:29:53,590 copying build/lib/lm_eval/tasks/afrimgsm/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T14:29:53,593 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa 2026-05-11T14:29:53,594 copying build/lib/lm_eval/tasks/openbookqa/openbookqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openbookqa 2026-05-11T14:29:53,596 copying build/lib/lm_eval/tasks/openbookqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openbookqa 2026-05-11T14:29:53,598 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,599 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,601 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,603 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,605 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,607 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,608 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,610 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,612 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,614 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,615 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,617 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,619 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,620 copying build/lib/lm_eval/tasks/crows_pairs/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,623 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,624 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,626 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,628 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,629 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,631 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,633 copying build/lib/lm_eval/tasks/crows_pairs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,635 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,637 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,638 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,640 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T14:29:53,642 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc 2026-05-11T14:29:53,644 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,645 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/_sciknoweval_mcqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,647 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Material.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,649 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,651 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,652 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/_var5shots_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,654 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,656 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T14:29:53,658 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:29:53,659 copying build/lib/lm_eval/tasks/e2lmc/mmlu_early_training/mmlu_early_training.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:29:53,661 copying build/lib/lm_eval/tasks/e2lmc/mmlu_early_training/custom_metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:29:53,663 copying build/lib/lm_eval/tasks/e2lmc/mmlu_early_training/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T14:29:53,665 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,666 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,668 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,670 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,671 copying build/lib/lm_eval/tasks/e2lmc/noor/_noor_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,673 copying build/lib/lm_eval/tasks/e2lmc/noor/_noor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,675 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,677 copying build/lib/lm_eval/tasks/e2lmc/noor/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,678 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,680 copying build/lib/lm_eval/tasks/e2lmc/noor/_noor_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,682 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,684 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,685 copying build/lib/lm_eval/tasks/e2lmc/noor/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,688 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,689 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,691 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,693 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,694 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T14:29:53,697 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa 2026-05-11T14:29:53,698 copying build/lib/lm_eval/tasks/headqa/headqa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/headqa 2026-05-11T14:29:53,700 copying build/lib/lm_eval/tasks/headqa/headqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/headqa 2026-05-11T14:29:53,702 copying build/lib/lm_eval/tasks/headqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/headqa 2026-05-11T14:29:53,704 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/libra 2026-05-11T14:29:53,705 copying build/lib/lm_eval/tasks/libra/librusec_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,707 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,709 copying build/lib/lm_eval/tasks/libra/matreshka_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,711 copying build/lib/lm_eval/tasks/libra/ru_quality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,713 copying build/lib/lm_eval/tasks/libra/ru_2wikimultihopqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,714 copying build/lib/lm_eval/tasks/libra/matreshka_yes_no.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,716 copying build/lib/lm_eval/tasks/libra/_complex_reasoning_and_mathematical_problems.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,718 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,719 copying build/lib/lm_eval/tasks/libra/passkey.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,721 copying build/lib/lm_eval/tasks/libra/ru_qasper.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,723 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,724 copying build/lib/lm_eval/tasks/libra/_question_answering_and_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,726 copying build/lib/lm_eval/tasks/libra/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,728 copying build/lib/lm_eval/tasks/libra/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,729 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,731 copying build/lib/lm_eval/tasks/libra/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,733 copying build/lib/lm_eval/tasks/libra/librusec_mhqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,735 copying build/lib/lm_eval/tasks/libra/ru_sci_abstract_retrieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,737 copying build/lib/lm_eval/tasks/libra/long_context_multiq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,739 copying build/lib/lm_eval/tasks/libra/ru_gsm100.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,741 copying build/lib/lm_eval/tasks/libra/passkey_with_librusec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,743 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,744 copying build/lib/lm_eval/tasks/libra/_simple_information_retrieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,746 copying build/lib/lm_eval/tasks/libra/ru_sci_passage_count.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,748 copying build/lib/lm_eval/tasks/libra/_multi_hop_question_answering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T14:29:53,750 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light 2026-05-11T14:29:53,752 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:29:53,753 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:29:53,755 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_exams_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:29:53,757 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_leaderboard_arabic_exams_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T14:29:53,759 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light 2026-05-11T14:29:53,762 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,763 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,764 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,766 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,768 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_msa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,770 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_dialects_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,771 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_sentiment_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,773 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,775 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_mcq_exams_test_ar_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,776 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,778 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,780 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T14:29:53,783 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:29:53,784 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_mt_sciq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:29:53,786 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_leaderboard_arabic_mt_sciq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:29:53,787 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T14:29:53,790 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:29:53,791 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_leaderboard_arabic_mt_boolq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:29:53,793 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_mt_boolq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:29:53,794 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T14:29:53,797 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:29:53,798 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_leaderboard_arabic_mt_piqa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:29:53,800 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_mt_piqa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:29:53,801 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T14:29:53,804 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:29:53,805 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_mt_openbook_qa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:29:53,807 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_leaderboard_arabic_mt_openbook_qa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:29:53,809 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T14:29:53,813 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,814 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_scenarios_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,816 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_sociology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,817 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_computer_science_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,819 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_virology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,821 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_us_foreign_policy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,823 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,824 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_sexuality_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,826 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_computer_science_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,828 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_abstract_algebra_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,829 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_international_law_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,831 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_physics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,833 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_management_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,835 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_chemistry_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,837 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_geography_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,838 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_medical_genetics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,840 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_nutrition_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,842 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_electrical_engineering_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,844 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_biology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,846 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_law_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,848 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_logical_fallacies_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,849 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_world_religions_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,851 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_psychology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,854 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_mathematics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,855 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_disputes_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,857 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_miscellaneous_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,859 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_business_ethics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,861 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_anatomy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,862 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_psychology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,864 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,866 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_accounting_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,868 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_philosophy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,869 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_world_history_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,871 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,873 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_global_facts_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,874 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_prehistory_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,876 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_marketing_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,878 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_astronomy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,879 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_aging_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,881 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_european_history_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,883 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_formal_logic_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,885 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_public_relations_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,887 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_security_studies_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,888 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_medicine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,890 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_machine_learning_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,892 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,893 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_statistics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,895 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_clinical_knowledge_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,897 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_biology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,899 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_mathematics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,900 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_conceptual_physics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,902 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_microeconomics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,904 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_physics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,906 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_jurisprudence_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,908 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_us_history_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,910 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_chemistry_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,911 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_elementary_mathematics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,913 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_econometrics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,915 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_medicine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,917 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_computer_security_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T14:29:53,919 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:29:53,920 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arbic_leaderboard_arabic_mt_copa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:29:53,922 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arabic_mt_copa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:29:53,923 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T14:29:53,926 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:29:53,927 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_leaderboard_arabic_mt_arc_easy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:29:53,929 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_mt_arc_easy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:29:53,931 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T14:29:53,932 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light 2026-05-11T14:29:53,935 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:29:53,936 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_mt_mmlu_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:29:53,938 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:29:53,940 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_leaderboard_arabic_mt_mmlu_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T14:29:53,942 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:29:53,943 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:29:53,945 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_leaderboard_arabic_mt_toxigen_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:29:53,947 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_mt_toxigen_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T14:29:53,949 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:29:53,950 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_mt_arc_challenge_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:29:53,952 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:29:53,954 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_leaderboard_arabic_mt_arc_challenge_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T14:29:53,957 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,958 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Syria_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,960 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_daily_life_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,962 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Astronomy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,964 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arab_Empire_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,965 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Somalia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,967 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Language_Origin_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,969 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromChina_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,971 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Art_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,973 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromAncientEgypt_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,975 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Math_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,977 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromRome_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,979 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_branches_and_schools_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,980 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Lebanon_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,982 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Culture_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,984 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ornament_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,985 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Oman_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,987 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Philosophy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,989 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islamic_law_system_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,991 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Algeria_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,992 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Qatar_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,994 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Kuwait_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,996 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Clothing_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:53,998 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Ancient_Egypt_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,000 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_History_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,001 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,003 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mauritania_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,005 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Funeral_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,007 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ceremony_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,008 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Architecture_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,010 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Egypt_modern_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,012 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_communication_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,014 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Palestine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,015 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromIslam_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,017 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Wedding_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,019 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,021 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_computer_and_phone_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,022 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Tunisia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,024 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_United_Arab_Emirates_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,026 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Geography_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,028 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromPersia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,030 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mesopotamia_civilization_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,031 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Calligraphy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,033 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_Education_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,035 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Music_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,037 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Saudi_Arabia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,038 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromByzantium_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,040 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Medicine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,042 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromGreece_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,044 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Sudan_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,045 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Comoros_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,047 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Bahrain_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,049 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Morocco_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,051 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,053 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Libya_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,054 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Iraq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,056 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Yemen_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,058 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Literature_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,060 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Food_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,061 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_entertainment_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,063 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Jordan_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T14:29:54,065 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:29:54,066 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:29:54,068 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_leaderboard_arabic_mt_race_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:29:54,070 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_mt_race_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T14:29:54,072 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:29:54,073 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_leaderboard_arabic_mt_hellaswag_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:29:54,075 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:29:54,077 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_mt_hellaswag_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T14:29:54,079 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/slr_bench 2026-05-11T14:29:54,080 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,082 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,084 copying build/lib/lm_eval/tasks/slr_bench/lm_eval_slr_bench.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,087 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,089 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,090 copying build/lib/lm_eval/tasks/slr_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,093 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_basic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,094 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,096 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T14:29:54,099 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2 2026-05-11T14:29:54,100 copying build/lib/lm_eval/tasks/logiqa2/logieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T14:29:54,102 copying build/lib/lm_eval/tasks/logiqa2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T14:29:54,104 copying build/lib/lm_eval/tasks/logiqa2/logiqa2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T14:29:54,106 copying build/lib/lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T14:29:54,110 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/longbench 2026-05-11T14:29:54,111 copying build/lib/lm_eval/tasks/longbench/repobench-p.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,112 copying build/lib/lm_eval/tasks/longbench/2wikimqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,114 copying build/lib/lm_eval/tasks/longbench/_longbench_multi_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,116 copying build/lib/lm_eval/tasks/longbench/musique.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,118 copying build/lib/lm_eval/tasks/longbench/_longbench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,119 copying build/lib/lm_eval/tasks/longbench/_longbench_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,121 copying build/lib/lm_eval/tasks/longbench/multifieldqa_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,123 copying build/lib/lm_eval/tasks/longbench/lcc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,125 copying build/lib/lm_eval/tasks/longbench/_longbench_code.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,126 copying build/lib/lm_eval/tasks/longbench/gov_report_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,128 copying build/lib/lm_eval/tasks/longbench/triviaqa_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,130 copying build/lib/lm_eval/tasks/longbench/passage_retrieval_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,132 copying build/lib/lm_eval/tasks/longbench/qasper.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,133 copying build/lib/lm_eval/tasks/longbench/qmsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,135 copying build/lib/lm_eval/tasks/longbench/_longbench_synthetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,137 copying build/lib/lm_eval/tasks/longbench/2wikimqa_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,138 copying build/lib/lm_eval/tasks/longbench/multifieldqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,140 copying build/lib/lm_eval/tasks/longbench/multi_news_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,142 copying build/lib/lm_eval/tasks/longbench/_longbench_fewshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,144 copying build/lib/lm_eval/tasks/longbench/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,146 copying build/lib/lm_eval/tasks/longbench/_longbench_single_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,148 copying build/lib/lm_eval/tasks/longbench/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,150 copying build/lib/lm_eval/tasks/longbench/triviaqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,152 copying build/lib/lm_eval/tasks/longbench/_longbench_fewshot_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,153 copying build/lib/lm_eval/tasks/longbench/hotpotqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,155 copying build/lib/lm_eval/tasks/longbench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,157 copying build/lib/lm_eval/tasks/longbench/trec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,159 copying build/lib/lm_eval/tasks/longbench/passage_retrieval_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,160 copying build/lib/lm_eval/tasks/longbench/passage_retrieval_en_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,162 copying build/lib/lm_eval/tasks/longbench/passage_count_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,164 copying build/lib/lm_eval/tasks/longbench/_longbench_code_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,165 copying build/lib/lm_eval/tasks/longbench/repobench-p_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,167 copying build/lib/lm_eval/tasks/longbench/lcc_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,169 copying build/lib/lm_eval/tasks/longbench/multifieldqa_en_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,170 copying build/lib/lm_eval/tasks/longbench/samsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,172 copying build/lib/lm_eval/tasks/longbench/narrativeqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,174 copying build/lib/lm_eval/tasks/longbench/_longbench_summarization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,176 copying build/lib/lm_eval/tasks/longbench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,178 copying build/lib/lm_eval/tasks/longbench/gov_report.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,179 copying build/lib/lm_eval/tasks/longbench/qasper_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,181 copying build/lib/lm_eval/tasks/longbench/lsht.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,183 copying build/lib/lm_eval/tasks/longbench/dureader.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,185 copying build/lib/lm_eval/tasks/longbench/vcsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,187 copying build/lib/lm_eval/tasks/longbench/_longbench_summarization_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,189 copying build/lib/lm_eval/tasks/longbench/_longbench_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,191 copying build/lib/lm_eval/tasks/longbench/trec_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,192 copying build/lib/lm_eval/tasks/longbench/multi_news.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,194 copying build/lib/lm_eval/tasks/longbench/_longbench_synthetic_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,196 copying build/lib/lm_eval/tasks/longbench/passage_count.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,197 copying build/lib/lm_eval/tasks/longbench/_longbench_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,199 copying build/lib/lm_eval/tasks/longbench/samsum_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,201 copying build/lib/lm_eval/tasks/longbench/hotpotqa_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T14:29:54,203 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv 2026-05-11T14:29:54,204 copying build/lib/lm_eval/tasks/asdiv/asdiv-cot-llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/asdiv 2026-05-11T14:29:54,206 copying build/lib/lm_eval/tasks/asdiv/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/asdiv 2026-05-11T14:29:54,207 copying build/lib/lm_eval/tasks/asdiv/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/asdiv 2026-05-11T14:29:54,210 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2026-05-11T14:29:54,211 copying build/lib/lm_eval/tasks/basqueglue/coref.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,213 copying build/lib/lm_eval/tasks/basqueglue/qnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,214 copying build/lib/lm_eval/tasks/basqueglue/bec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,216 copying build/lib/lm_eval/tasks/basqueglue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,218 copying build/lib/lm_eval/tasks/basqueglue/vaxx.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,220 copying build/lib/lm_eval/tasks/basqueglue/bhtc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,222 copying build/lib/lm_eval/tasks/basqueglue/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,223 copying build/lib/lm_eval/tasks/basqueglue/wic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T14:29:54,226 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cruxeval 2026-05-11T14:29:54,227 copying build/lib/lm_eval/tasks/cruxeval/cruxeval_output_cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,229 copying build/lib/lm_eval/tasks/cruxeval/cruxeval_output.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,230 copying build/lib/lm_eval/tasks/cruxeval/cruxeval_input_08.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,232 copying build/lib/lm_eval/tasks/cruxeval/cruxeval_input.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,234 copying build/lib/lm_eval/tasks/cruxeval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,236 copying build/lib/lm_eval/tasks/cruxeval/cruxeval_input_cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,238 copying build/lib/lm_eval/tasks/cruxeval/cruxeval_output_08.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,239 copying build/lib/lm_eval/tasks/cruxeval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,242 copying build/lib/lm_eval/tasks/cruxeval/cruxeval_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cruxeval 2026-05-11T14:29:54,244 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa 2026-05-11T14:29:54,245 copying build/lib/lm_eval/tasks/mathqa/mathqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mathqa 2026-05-11T14:29:54,247 copying build/lib/lm_eval/tasks/mathqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mathqa 2026-05-11T14:29:54,249 copying build/lib/lm_eval/tasks/mathqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mathqa 2026-05-11T14:29:54,252 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa 2026-05-11T14:29:54,253 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/copa_ar 2026-05-11T14:29:54,254 copying build/lib/lm_eval/tasks/alghafa/copa_ar/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/copa_ar 2026-05-11T14:29:54,256 copying build/lib/lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/copa_ar 2026-05-11T14:29:54,258 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T14:29:54,259 copying build/lib/lm_eval/tasks/alghafa/piqa_ar/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/piqa_ar 2026-05-11T14:29:54,261 copying build/lib/lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/piqa_ar 2026-05-11T14:29:54,263 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T14:29:54,265 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,266 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,268 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,270 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,271 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,273 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,275 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,277 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,278 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,280 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,282 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/_default_template_spanish_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,284 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,285 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,287 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,289 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,291 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,293 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,294 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,296 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,298 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,299 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,301 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,303 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,304 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,306 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,308 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,309 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,311 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,313 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,314 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,316 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,318 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,320 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,321 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,323 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,325 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,326 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,328 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,330 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,332 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,333 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,335 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,337 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,339 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,340 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,342 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,344 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,345 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,347 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,349 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,351 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,353 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,355 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,357 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,358 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,360 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,362 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,364 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,365 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,367 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T14:29:54,369 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish 2026-05-11T14:29:54,371 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/mmlu-redux-2.0-spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish 2026-05-11T14:29:54,373 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench 2026-05-11T14:29:54,374 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,375 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_mac.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,377 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_myc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,379 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,381 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,383 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_electrom.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,385 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msac.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,386 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,388 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/default_darija_sentiment_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,390 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T14:29:54,392 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:54,393 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/summarization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:54,396 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/summarization_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:54,398 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:54,399 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:54,401 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/summarization_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T14:29:54,404 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,405 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_en_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,408 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,409 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/translation_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,411 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_fr_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,413 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,414 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,416 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_fr_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,418 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,419 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,421 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,423 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,425 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,427 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_dr_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,428 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_msa_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,430 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,432 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_msa_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,434 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,435 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,437 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,439 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,441 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_msa_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,442 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_en_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,444 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,446 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,448 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,449 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,451 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_dr_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,453 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_en_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,454 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,456 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,458 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,459 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T14:29:54,461 copying build/lib/lm_eval/tasks/darija_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench 2026-05-11T14:29:54,464 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,465 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/translation_dr_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,467 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/transliteration_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,469 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/transliteration_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,470 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,472 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/translation_ar_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,474 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,476 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/transliteration_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T14:29:54,478 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa 2026-05-11T14:29:54,479 copying build/lib/lm_eval/tasks/siqa/siqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/siqa 2026-05-11T14:29:54,481 copying build/lib/lm_eval/tasks/siqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/siqa 2026-05-11T14:29:54,484 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag 2026-05-11T14:29:54,485 copying build/lib/lm_eval/tasks/hellaswag/hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hellaswag 2026-05-11T14:29:54,486 copying build/lib/lm_eval/tasks/hellaswag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hellaswag 2026-05-11T14:29:54,488 copying build/lib/lm_eval/tasks/hellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hellaswag 2026-05-11T14:29:54,492 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,493 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,495 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,496 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,498 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,499 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,501 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,503 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,505 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,506 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,508 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,510 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,511 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__plur_MS_ANPHR.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,513 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,515 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,516 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,518 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,520 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__sing_MS_MV_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,522 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,523 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,525 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,527 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,529 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,530 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,532 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,533 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,535 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,537 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,538 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,540 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,542 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,544 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,545 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,547 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,549 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,550 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,552 copying build/lib/lm_eval/tasks/lm_syneval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,555 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__plur_MS_MV_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,557 copying build/lib/lm_eval/tasks/lm_syneval/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,558 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,560 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__sing_MS_ANPHR.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,562 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,564 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,565 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,567 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,569 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,571 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,572 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,574 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,576 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,577 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,579 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,581 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,583 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,584 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,586 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,588 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,589 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__plur_MS_LMV_LMV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,591 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,593 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__sing_MS_LMV_LMV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,595 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,597 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,598 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,600 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,602 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,603 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,605 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,607 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,608 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,610 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,612 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,613 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,616 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,617 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,619 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,621 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T14:29:54,625 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,625 copying build/lib/lm_eval/tasks/blimp_nl/adverbial_modification__position_type.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,627 copying build/lib/lm_eval/tasks/blimp_nl/extraposition__argument_nominal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,629 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,631 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,633 copying build/lib/lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,635 copying build/lib/lm_eval/tasks/blimp_nl/passive__impersonal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,636 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,638 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,640 copying build/lib/lm_eval/tasks/blimp_nl/relativization__island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,641 copying build/lib/lm_eval/tasks/blimp_nl/determiners__geen_expletive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,643 copying build/lib/lm_eval/tasks/blimp_nl/binding_principle_a__c_command.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,645 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,646 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,648 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__argument_number_in_transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,650 copying build/lib/lm_eval/tasks/blimp_nl/complementive__ditransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,652 copying build/lib/lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,653 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__perception_dat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,655 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,657 copying build/lib/lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,659 copying build/lib/lm_eval/tasks/blimp_nl/extraposition__adjectival_supplementive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,661 copying build/lib/lm_eval/tasks/blimp_nl/extraposition__adjectival_adverbial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,663 copying build/lib/lm_eval/tasks/blimp_nl/binding_principle_a__monomorphemic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,665 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__stranding_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,667 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,669 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__order_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,670 copying build/lib/lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,672 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,675 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,676 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,679 copying build/lib/lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,680 copying build/lib/lm_eval/tasks/blimp_nl/adpositional_phrases__argument_scrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,682 copying build/lib/lm_eval/tasks/blimp_nl/verb_second__order_main.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,684 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,685 copying build/lib/lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,687 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,689 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__resumptive_prolepsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,690 copying build/lib/lm_eval/tasks/blimp_nl/crossing_dependencies__cross_dependency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,692 copying build/lib/lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,693 copying build/lib/lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,695 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,697 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,699 copying build/lib/lm_eval/tasks/blimp_nl/blimp_nl_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,701 copying build/lib/lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,703 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,705 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,707 copying build/lib/lm_eval/tasks/blimp_nl/complementive__position_adverb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,708 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,710 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,712 copying build/lib/lm_eval/tasks/blimp_nl/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,714 copying build/lib/lm_eval/tasks/blimp_nl/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,716 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,717 copying build/lib/lm_eval/tasks/blimp_nl/anaphor_agreement__person.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,719 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,721 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__superiority.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,723 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,724 copying build/lib/lm_eval/tasks/blimp_nl/adverbial_modification__position_proform.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,726 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,728 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,730 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,731 copying build/lib/lm_eval/tasks/blimp_nl/adpositional_phrases__argument_r_extraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,733 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,734 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,736 copying build/lib/lm_eval/tasks/blimp_nl/passive__aci.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,738 copying build/lib/lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,739 copying build/lib/lm_eval/tasks/blimp_nl/r_words__adverbial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,741 copying build/lib/lm_eval/tasks/blimp_nl/complementive__intransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,742 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,744 copying build/lib/lm_eval/tasks/blimp_nl/determiners__geen_scrambling_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,746 copying build/lib/lm_eval/tasks/blimp_nl/complementive__position_verb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,747 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__argument_number_ditransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,749 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__perfect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,751 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__stranding_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,752 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,754 copying build/lib/lm_eval/tasks/blimp_nl/anaphor_agreement__number.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,756 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,758 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__order_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,760 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,761 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,763 copying build/lib/lm_eval/tasks/blimp_nl/determiners__negative_polarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,765 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,766 copying build/lib/lm_eval/tasks/blimp_nl/determiners__geen_scrambling_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,768 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,770 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__complementizer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,772 copying build/lib/lm_eval/tasks/blimp_nl/complementive__transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,773 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,775 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,777 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T14:29:54,779 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench 2026-05-11T14:29:54,781 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc 2026-05-11T14:29:54,782 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:54,783 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:54,785 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:54,787 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:54,789 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T14:29:54,791 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:54,792 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:54,794 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:54,796 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:54,797 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T14:29:54,800 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:54,801 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:54,803 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:54,804 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:54,806 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T14:29:54,808 copying build/lib/lm_eval/tasks/afrobench/naijarc/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc 2026-05-11T14:29:54,809 copying build/lib/lm_eval/tasks/afrobench/naijarc/naijarc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc 2026-05-11T14:29:54,811 copying build/lib/lm_eval/tasks/afrobench/naijarc/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc 2026-05-11T14:29:54,814 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:54,815 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:54,816 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:54,818 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:54,820 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T14:29:54,822 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:54,823 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:54,824 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:54,826 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:54,828 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T14:29:54,830 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T14:29:54,831 copying build/lib/lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T14:29:54,833 copying build/lib/lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench_lite.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T14:29:54,835 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex 2026-05-11T14:29:54,837 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_1 2026-05-11T14:29:54,839 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,840 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,842 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,843 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,845 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,847 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,849 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,851 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,852 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,854 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,856 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,858 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,859 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,861 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,863 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,864 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,866 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,868 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,869 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,871 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,873 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,875 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,876 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,878 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,880 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,882 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,884 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,885 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,887 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,889 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,890 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,892 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,894 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,895 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T14:29:54,898 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,899 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,901 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,903 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,905 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,906 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,908 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,910 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,912 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,914 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,915 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,917 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,919 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,921 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,923 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,925 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,927 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,928 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,930 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,932 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,934 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,936 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,938 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,940 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,941 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,943 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,945 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,947 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,949 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,950 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,952 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,954 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,956 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,957 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T14:29:54,960 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_2 2026-05-11T14:29:54,961 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,962 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,964 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,966 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,968 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,970 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,971 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,973 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,975 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,977 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,978 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,980 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,982 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,984 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,985 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,987 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,989 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,991 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,992 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,994 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,996 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,997 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:54,999 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,001 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,002 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,004 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,006 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,007 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,009 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,011 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,013 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,014 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,016 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,018 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T14:29:55,021 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,022 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,024 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,025 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,027 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,029 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,030 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,032 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,034 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,035 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,037 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,039 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,041 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,042 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,044 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,046 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,048 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,050 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,052 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,054 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,057 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,059 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,062 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,063 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,065 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,067 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,068 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,070 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,072 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,073 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,075 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,077 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,079 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,081 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T14:29:55,082 copying build/lib/lm_eval/tasks/afrobench/ntrex/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex 2026-05-11T14:29:55,084 copying build/lib/lm_eval/tasks/afrobench/ntrex/ntrex.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex 2026-05-11T14:29:55,088 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_3 2026-05-11T14:29:55,090 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,092 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,094 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,096 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,098 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,099 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,101 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,103 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,104 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,106 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,108 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,109 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,111 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,113 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,115 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,116 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,118 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,120 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,121 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,123 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,125 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,127 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,129 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,131 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,132 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,134 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,136 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,138 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,139 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,141 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,143 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,145 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,146 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,148 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T14:29:55,151 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,152 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,154 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,155 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,157 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,159 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,161 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,163 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,165 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,166 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,168 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,170 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,172 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,173 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,175 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,177 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,179 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,180 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,182 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,184 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,185 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,187 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,189 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,191 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,192 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,194 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,196 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,198 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,199 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,201 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,203 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,204 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,206 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,208 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T14:29:55,209 copying build/lib/lm_eval/tasks/afrobench/ntrex/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex 2026-05-11T14:29:55,212 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:55,214 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,215 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,217 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,219 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,220 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,222 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,224 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,226 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,228 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,230 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,232 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/run.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,234 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,236 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,238 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,240 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,242 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,244 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,245 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,248 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T14:29:55,250 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,251 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,253 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,255 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,257 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,258 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,260 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,262 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,264 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,266 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,268 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,270 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,272 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,273 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,275 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,277 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,279 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,280 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T14:29:55,282 copying build/lib/lm_eval/tasks/afrobench/afrisenti/fewshot.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:55,285 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,286 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,288 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,289 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,291 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,293 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,294 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,296 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,298 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,300 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,301 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/run.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,303 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,305 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,307 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,309 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,311 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,312 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,314 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,316 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T14:29:55,318 copying build/lib/lm_eval/tasks/afrobench/afrisenti/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:55,319 copying build/lib/lm_eval/tasks/afrobench/afrisenti/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:55,322 copying build/lib/lm_eval/tasks/afrobench/afrisenti/afrisenti.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T14:29:55,324 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,325 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,327 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,328 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,330 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,332 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,333 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,335 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,337 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,338 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,340 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,342 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,344 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,345 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,347 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,349 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,351 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,352 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T14:29:55,355 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,356 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,358 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,359 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,361 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,363 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,364 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,366 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,368 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,369 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,371 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,373 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,374 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,376 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,378 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,379 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,381 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,383 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T14:29:55,385 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum 2026-05-11T14:29:55,387 copying build/lib/lm_eval/tasks/afrobench/xlsum/xlsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum 2026-05-11T14:29:55,389 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,390 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,392 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_pidgin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,394 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,396 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_igbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,398 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_oromo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,400 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_hausa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,402 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_kirundi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,404 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,406 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,408 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_tigrinya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,409 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_somali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,411 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,413 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_yoruba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,414 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_amharic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T14:29:55,417 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,418 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,420 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_pidgin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,421 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,423 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_igbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,425 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_oromo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,427 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_hausa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,429 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_kirundi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,431 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,434 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,437 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_tigrinya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,438 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_somali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,440 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,442 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_yoruba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,444 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_amharic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T14:29:55,445 copying build/lib/lm_eval/tasks/afrobench/xlsum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum 2026-05-11T14:29:55,447 copying build/lib/lm_eval/tasks/afrobench/xlsum/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum 2026-05-11T14:29:55,450 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,451 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,453 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_pidgin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,454 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,456 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_igbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,457 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_oromo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,459 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_hausa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,461 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_kirundi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,463 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,465 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,466 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_tigrinya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,468 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_somali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,470 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,472 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_yoruba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,473 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_amharic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T14:29:55,476 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:29:55,477 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:55,478 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:55,480 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:55,482 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:55,484 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T14:29:55,486 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:55,487 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:55,489 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:55,491 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:55,492 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T14:29:55,495 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:55,496 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:55,497 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:55,499 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:55,501 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T14:29:55,502 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/openai_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:29:55,504 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:29:55,506 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T14:29:55,508 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:55,509 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:55,511 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:55,513 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:55,514 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T14:29:55,517 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:55,518 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:55,520 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:55,521 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:55,523 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T14:29:55,525 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:29:55,527 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,528 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,530 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,532 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,534 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,536 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,537 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,539 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,541 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,543 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T14:29:55,545 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,546 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,548 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,550 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,551 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,553 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,555 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,556 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,558 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,560 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T14:29:55,563 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,564 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,566 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,568 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,570 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,572 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,574 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,576 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,578 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,579 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T14:29:55,581 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:29:55,582 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/uhura.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:29:55,584 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T14:29:55,587 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,588 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,590 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,591 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,593 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,595 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,596 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,598 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,600 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,602 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T14:29:55,604 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,605 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,607 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,609 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,611 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,613 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,615 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,616 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,618 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,620 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T14:29:55,622 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:29:55,623 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,625 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,627 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,628 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,630 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,632 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,634 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,636 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,637 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,639 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,641 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,643 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,644 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,646 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,648 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,650 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,651 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,653 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,655 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,656 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,658 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,660 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,661 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T14:29:55,664 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,665 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,667 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,669 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,670 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,672 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,674 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,676 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,678 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,679 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,681 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,683 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,685 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,687 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,688 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,690 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,692 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,693 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,695 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,697 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,698 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,700 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,702 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T14:29:55,704 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,705 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,707 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,709 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,711 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,713 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,715 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,717 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,719 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,721 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,723 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,724 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,726 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,728 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,730 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,731 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,733 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,735 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,736 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,738 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,740 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,741 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,743 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T14:29:55,745 copying build/lib/lm_eval/tasks/afrobench/masakhapos/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:29:55,747 copying build/lib/lm_eval/tasks/afrobench/masakhapos/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:29:55,749 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,750 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,752 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,754 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,756 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,758 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,760 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,762 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,763 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,765 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,767 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,769 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,771 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,772 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,774 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,775 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,777 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,779 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,780 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,782 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,784 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,786 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,788 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T14:29:55,790 copying build/lib/lm_eval/tasks/afrobench/masakhapos/masakhapos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:29:55,793 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,794 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,796 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,798 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,800 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,802 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,804 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,806 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,808 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,810 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,812 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,814 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,816 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,817 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,819 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,821 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,822 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,824 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,826 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,828 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,829 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,831 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,833 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T14:29:55,835 copying build/lib/lm_eval/tasks/afrobench/masakhapos/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T14:29:55,838 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores 2026-05-11T14:29:55,839 copying build/lib/lm_eval/tasks/afrobench/flores/flores.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores 2026-05-11T14:29:55,841 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_1 2026-05-11T14:29:55,843 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,845 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,847 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,849 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,850 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,852 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,854 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lua_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,855 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-twi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,857 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,859 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,860 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,862 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,867 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,868 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kab_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,870 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,872 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,874 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,876 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,878 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,879 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,881 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-mos_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,883 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,885 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,886 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-umb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,888 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,890 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,892 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,894 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,895 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,897 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sag_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,899 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,901 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,903 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aka_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,904 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,906 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,908 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,910 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,912 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,914 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,915 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,917 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-run_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,919 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,921 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,923 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,924 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,926 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,928 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,930 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,931 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,933 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,935 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,937 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,940 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ban_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,942 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,945 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,947 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,949 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,951 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,952 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,954 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tum_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,956 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,958 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,960 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-acq_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,961 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T14:29:55,963 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1 2026-05-11T14:29:55,967 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,968 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,970 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,972 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,973 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tum_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,975 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,977 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,979 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aka_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,980 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,982 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,984 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,985 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,987 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,989 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kab_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,991 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sun_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,992 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ban_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,994 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,996 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,998 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:55,999 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_mos_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,001 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_plt_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,002 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,004 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,006 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,008 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,009 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,011 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,013 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sag_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,015 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,016 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lug_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,018 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_acq_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,020 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,022 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lua_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,023 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,025 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,027 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,030 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,035 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,040 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_arz_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,044 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_run_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,046 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_swh_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,048 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,049 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,051 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kea_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,053 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,055 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,057 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_twi_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,058 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,060 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,062 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,064 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,065 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,067 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,069 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_luo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,071 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,073 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,074 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sot_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,076 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,078 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ary_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,080 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,081 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,083 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,085 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_umb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,088 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nus_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T14:29:56,090 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_2 2026-05-11T14:29:56,092 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,094 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,095 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,097 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,099 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,101 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,103 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lua_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,105 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-twi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,106 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,108 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,110 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,111 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,113 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,114 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kab_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,116 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,118 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,120 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,122 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,123 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,125 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,127 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-mos_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,129 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,131 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,133 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-umb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,134 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,136 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,138 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,140 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,141 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,143 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sag_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,145 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,147 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,149 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aka_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,151 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,153 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,155 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,157 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,158 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,160 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,162 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,164 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-run_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,166 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,168 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,170 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,171 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,173 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,175 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,176 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,178 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,180 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,181 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,185 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,186 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ban_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,188 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,190 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,192 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,193 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,195 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,197 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,199 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tum_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,201 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,202 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,204 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-acq_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,206 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T14:29:56,208 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2 2026-05-11T14:29:56,211 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,212 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,215 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,217 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,219 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tum_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,221 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,223 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,225 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aka_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,227 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,229 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,232 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,234 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,236 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,238 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kab_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,241 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sun_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,243 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ban_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,245 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,247 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,250 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,252 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_mos_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,254 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_plt_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,256 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,258 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,260 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,263 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,265 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,268 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,270 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sag_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,272 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,275 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lug_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,277 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_acq_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,280 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,282 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lua_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,284 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,287 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,290 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,292 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,295 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,297 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_arz_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,299 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_run_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,301 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_swh_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,303 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,305 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,308 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kea_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,310 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,312 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,314 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_twi_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,316 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,318 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,321 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,323 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,325 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,327 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,329 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_luo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,331 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,332 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,334 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sot_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,335 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,337 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ary_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,339 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,340 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,342 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,344 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_umb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,346 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nus_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T14:29:56,348 copying build/lib/lm_eval/tasks/afrobench/flores/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores 2026-05-11T14:29:56,350 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_3 2026-05-11T14:29:56,352 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,353 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,355 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,357 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,359 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,361 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,362 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lua_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,364 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-twi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,366 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,368 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,369 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,371 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,373 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,374 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kab_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,376 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,378 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,380 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,381 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,383 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,385 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,387 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-mos_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,388 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,390 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,392 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-umb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,394 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,396 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,398 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,399 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,401 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,403 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sag_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,405 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,406 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,408 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aka_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,410 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,411 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,413 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,415 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,417 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,419 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,420 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,422 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-run_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,424 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,426 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,429 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,431 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,433 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,435 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,436 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,438 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,440 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,441 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,443 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,445 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ban_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,447 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,449 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,450 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,452 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,454 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,455 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,457 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tum_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,459 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,461 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,463 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-acq_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,465 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T14:29:56,466 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3 2026-05-11T14:29:56,469 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,470 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,472 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,474 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,475 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tum_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,477 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,479 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,481 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aka_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,482 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,484 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,486 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,488 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,489 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,491 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kab_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,493 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sun_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,495 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ban_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,497 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,499 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,500 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,502 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_mos_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,504 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_plt_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,505 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,507 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,509 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,511 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,513 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,515 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,516 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sag_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,518 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,520 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lug_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,522 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_acq_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,524 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,526 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lua_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,528 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,530 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,531 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,533 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,535 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,537 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_arz_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,538 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_run_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,540 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_swh_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,542 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,544 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,545 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kea_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,547 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,549 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,551 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_twi_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,553 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,555 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,557 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,559 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,561 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,562 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,564 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_luo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,566 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,568 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,570 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sot_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,571 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,573 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ary_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,575 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,576 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,578 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,580 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_umb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,582 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nus_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T14:29:56,584 copying build/lib/lm_eval/tasks/afrobench/flores/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores 2026-05-11T14:29:56,587 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr 2026-05-11T14:29:56,588 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:56,589 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:56,591 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:56,593 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:56,595 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:56,597 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:56,598 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T14:29:56,600 copying build/lib/lm_eval/tasks/afrobench/adr/afridiacritics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr 2026-05-11T14:29:56,602 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:56,603 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:56,605 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:56,607 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:56,609 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:56,610 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:56,612 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T14:29:56,614 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:56,615 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:56,617 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:56,619 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:56,621 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:56,622 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:56,624 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T14:29:56,626 copying build/lib/lm_eval/tasks/afrobench/adr/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr 2026-05-11T14:29:56,628 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:56,629 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:56,631 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:56,633 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:56,635 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:56,637 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:56,639 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T14:29:56,641 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:56,642 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:56,644 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:56,646 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:56,648 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:56,649 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:56,651 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T14:29:56,653 copying build/lib/lm_eval/tasks/afrobench/adr/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr 2026-05-11T14:29:56,655 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele 2026-05-11T14:29:56,657 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,658 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,660 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,661 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,663 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,665 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,666 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,668 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,670 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,672 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,673 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,675 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,677 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,678 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,680 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,682 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,684 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,686 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,687 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,689 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,691 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,692 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,694 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,696 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,698 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,700 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,702 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,704 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,706 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,707 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,709 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,711 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,712 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T14:29:56,715 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,716 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,718 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,720 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,721 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,723 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,725 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,727 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,728 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,730 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,732 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,733 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,735 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,737 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,739 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,740 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,742 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,744 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,745 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,747 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,749 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,751 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,753 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,755 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,757 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,759 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,761 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,762 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,764 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,766 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,768 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,770 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,771 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T14:29:56,774 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,775 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,776 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,778 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,780 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,782 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,783 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,785 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,787 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,789 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,791 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,792 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,794 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,796 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,798 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,799 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,801 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,803 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,805 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,806 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,808 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,810 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,812 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,813 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,815 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,817 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,818 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,820 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,822 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,823 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,825 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,827 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,829 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T14:29:56,830 copying build/lib/lm_eval/tasks/afrobench/belebele/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele 2026-05-11T14:29:56,832 copying build/lib/lm_eval/tasks/afrobench/belebele/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele 2026-05-11T14:29:56,835 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:56,837 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:56,839 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:56,841 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:56,842 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:56,844 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,067 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,069 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,071 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,073 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,076 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,078 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,080 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,082 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,084 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,086 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,089 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,091 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,093 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,095 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,097 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,099 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,101 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,104 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,106 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,108 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,110 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,112 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,115 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,117 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,119 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,121 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,124 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T14:29:57,127 copying build/lib/lm_eval/tasks/afrobench/belebele/belebele.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele 2026-05-11T14:29:57,130 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,132 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,134 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,137 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,139 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,141 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,143 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,146 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,148 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,150 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,153 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,155 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,157 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,159 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,161 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,164 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,166 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,168 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,170 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,172 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,174 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,176 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,179 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,181 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,183 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,185 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,187 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,189 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,191 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,193 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,195 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,198 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,200 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T14:29:57,203 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand 2026-05-11T14:29:57,205 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_1 2026-05-11T14:29:57,207 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,209 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,211 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,213 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,215 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,217 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,220 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,222 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,224 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,227 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,229 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,231 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,233 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,235 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,237 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,240 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,242 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,244 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,246 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,248 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,250 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,252 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,253 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,255 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T14:29:57,258 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,259 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_pcm-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,261 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ewe-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,262 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_swa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,264 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_luo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,266 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_xho-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,268 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_twi-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,269 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_sna-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,271 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bam-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,272 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bbj-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,274 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_lug-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,276 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_hau-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,277 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_yor-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,279 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_nya-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,281 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_kin-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,283 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,285 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_tsn-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,286 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_amh-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,288 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_wol-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,290 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_mos-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,292 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_zul-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,294 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,296 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ibo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,298 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_fon-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T14:29:57,300 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_2 2026-05-11T14:29:57,302 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,303 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,305 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,307 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,309 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,310 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,312 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,314 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,315 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,317 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,319 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,321 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,322 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,324 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,326 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,328 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,330 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,332 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,334 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,336 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,338 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,339 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,341 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,343 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T14:29:57,345 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,346 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_pcm-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,348 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ewe-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,350 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_swa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,351 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_luo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,353 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_xho-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,355 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_twi-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,357 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_sna-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,358 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bam-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,360 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bbj-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,362 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_lug-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,364 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_hau-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,365 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_yor-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,367 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_nya-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,369 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_kin-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,371 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,373 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_tsn-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,374 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_amh-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,376 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_wol-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,378 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_mos-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,379 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_zul-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,381 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,383 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ibo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,385 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_fon-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T14:29:57,386 copying build/lib/lm_eval/tasks/afrobench/mafand/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand 2026-05-11T14:29:57,389 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_3 2026-05-11T14:29:57,391 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,392 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,394 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,396 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,397 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,399 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,401 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,402 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,404 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,406 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,408 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,409 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,411 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,413 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,414 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,416 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,418 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,420 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,421 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,423 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,425 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,426 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,428 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,430 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T14:29:57,432 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,433 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_pcm-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,435 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ewe-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,437 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_swa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,439 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_luo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,441 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_xho-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,442 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_twi-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,444 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_sna-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,446 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bam-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,447 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bbj-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,449 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_lug-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,451 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_hau-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,452 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_yor-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,454 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_nya-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,456 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_kin-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,457 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,459 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_tsn-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,461 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_amh-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,463 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_wol-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,464 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_mos-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,466 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_zul-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,468 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,470 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ibo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,472 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_fon-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T14:29:57,473 copying build/lib/lm_eval/tasks/afrobench/mafand/mafand.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand 2026-05-11T14:29:57,475 copying build/lib/lm_eval/tasks/afrobench/mafand/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand 2026-05-11T14:29:57,478 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib 2026-05-11T14:29:57,480 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,481 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,483 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,485 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,487 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,488 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,490 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,491 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,493 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,495 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,497 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,499 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,501 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,502 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,504 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,506 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,508 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,510 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,512 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,513 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,515 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,517 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,519 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,521 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,523 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,524 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,526 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,528 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,529 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,531 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,533 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,534 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,536 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,538 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,539 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,541 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,543 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,544 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,546 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,548 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,550 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,552 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,554 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,555 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,557 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,559 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,561 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,563 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,564 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,566 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,568 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,569 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,571 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,573 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,574 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,576 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,578 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,579 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,581 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T14:29:57,584 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,585 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,587 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,589 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,590 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,592 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,594 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,596 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,597 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,599 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,601 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,603 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,604 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,606 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,608 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,610 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,612 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,613 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,615 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,617 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,619 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,620 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,622 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,624 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,625 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,627 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,629 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,630 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,632 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,634 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,636 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,637 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,659 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,662 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,664 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,667 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,669 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,671 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,673 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,676 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,678 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,680 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,682 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,684 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,686 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,688 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,691 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,693 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,695 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,697 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,699 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,701 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,703 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,705 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,708 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,710 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,711 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,714 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,715 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T14:29:57,719 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,720 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,722 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,724 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,726 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,728 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,730 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,732 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,734 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,735 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,737 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,739 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,741 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,743 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,745 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,746 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,748 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,750 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,752 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,754 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,756 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,758 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,760 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,762 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,764 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,765 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,768 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,769 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,771 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,773 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,775 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,777 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,779 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,781 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,783 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,785 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,787 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,789 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,791 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,793 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,794 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,796 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,798 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,800 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,801 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,804 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,805 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,807 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,809 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,811 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,813 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,815 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,817 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,819 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,821 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,823 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,824 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,826 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,828 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T14:29:57,830 copying build/lib/lm_eval/tasks/afrobench/sib/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib 2026-05-11T14:29:57,832 copying build/lib/lm_eval/tasks/afrobench/sib/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib 2026-05-11T14:29:57,836 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,837 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,839 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,841 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,843 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,845 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,847 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,849 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,851 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,853 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,855 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,857 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,859 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,861 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,863 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,865 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,867 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,869 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,871 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,873 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,875 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,877 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,879 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,880 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,882 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,884 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,886 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,887 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,889 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,891 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,893 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,895 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,897 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,899 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,900 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,902 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,904 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,906 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,908 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,909 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,911 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,913 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,915 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,917 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,919 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,921 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,922 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,924 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,926 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,928 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,930 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,931 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,933 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,935 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,937 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,939 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,940 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,942 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,944 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T14:29:57,947 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,948 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,950 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,952 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,954 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,956 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,958 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,961 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,963 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,965 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,968 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,970 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,972 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,974 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,976 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,978 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,980 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,982 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,985 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,987 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,989 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,991 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,993 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,996 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:57,998 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,000 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,002 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,004 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,006 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,009 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,011 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,013 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,015 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,017 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,020 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,022 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,024 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,026 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,028 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,030 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,032 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,035 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,037 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,039 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,041 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,043 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,046 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,049 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,051 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,054 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,056 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,058 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,060 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,062 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,064 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,066 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,068 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,070 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,072 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T14:29:58,075 copying build/lib/lm_eval/tasks/afrobench/sib/sib.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib 2026-05-11T14:29:58,077 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:29:58,079 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,081 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,083 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,085 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,087 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,089 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,091 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,093 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,095 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,097 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,100 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,102 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,104 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,106 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,108 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,110 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,112 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,115 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,117 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,119 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,122 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,124 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,126 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T14:29:58,129 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,130 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,132 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,134 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,136 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,138 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,140 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,141 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,143 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,145 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,147 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,149 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,150 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,152 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,154 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,155 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,157 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,160 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,161 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,163 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,165 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,167 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,168 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T14:29:58,171 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,172 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,173 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,175 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,177 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,179 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,181 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,182 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,184 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,186 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,188 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,190 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,192 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,194 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,195 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,197 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,199 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,201 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,203 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,205 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,207 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,209 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,210 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T14:29:58,212 copying build/lib/lm_eval/tasks/afrobench/masakhaner/masakhaner.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:29:58,214 copying build/lib/lm_eval/tasks/afrobench/masakhaner/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:29:58,217 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,218 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,220 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,222 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,224 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,225 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,227 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,229 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,231 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,233 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,234 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,236 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,238 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,240 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,241 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,243 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,245 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,247 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,249 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,251 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,252 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,254 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,255 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T14:29:58,258 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,259 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,260 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,262 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,264 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,266 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,268 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,269 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,271 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,273 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,274 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,276 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,278 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,280 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,282 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,284 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,286 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,289 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,291 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,292 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,294 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,296 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,298 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T14:29:58,299 copying build/lib/lm_eval/tasks/afrobench/masakhaner/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner 2026-05-11T14:29:58,302 copying build/lib/lm_eval/tasks/afrobench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench 2026-05-11T14:29:58,304 copying build/lib/lm_eval/tasks/afrobench/afrobench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench 2026-05-11T14:29:58,306 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti 2026-05-11T14:29:58,308 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,309 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,311 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,313 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,314 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,316 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,318 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,320 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T14:29:58,323 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,324 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,327 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,329 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,331 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,333 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,335 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,337 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T14:29:58,340 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,341 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,343 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,345 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,347 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,349 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,351 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,353 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T14:29:58,355 copying build/lib/lm_eval/tasks/afrobench/nollysenti/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti 2026-05-11T14:29:58,357 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,358 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,360 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,362 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,364 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,365 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,367 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,369 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T14:29:58,370 copying build/lib/lm_eval/tasks/afrobench/nollysenti/nollysenti.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti 2026-05-11T14:29:58,373 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,374 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,376 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,378 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,380 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,382 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,383 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,385 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T14:29:58,387 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa 2026-05-11T14:29:58,389 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,390 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,392 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,394 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,396 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,397 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,399 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,401 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,403 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,405 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,407 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,408 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T14:29:58,411 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,412 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,414 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,416 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,418 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,420 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,422 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,424 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,425 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,427 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,429 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,430 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T14:29:58,433 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,434 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,444 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,445 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,447 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,449 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,451 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,453 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,455 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,457 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,458 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,460 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T14:29:58,462 copying build/lib/lm_eval/tasks/afrobench/afriqa/afriqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa 2026-05-11T14:29:58,464 copying build/lib/lm_eval/tasks/afrobench/afriqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa 2026-05-11T14:29:58,466 copying build/lib/lm_eval/tasks/afrobench/afriqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa 2026-05-11T14:29:58,468 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,469 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,471 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,473 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,475 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,477 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,479 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,480 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,482 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,484 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,486 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,487 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T14:29:58,490 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,491 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,493 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,495 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,497 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,499 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,500 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,502 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,504 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,506 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,508 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,510 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T14:29:58,512 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent 2026-05-11T14:29:58,514 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,515 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,517 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,519 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,521 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,523 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,525 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,526 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,528 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,530 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,531 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,533 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,535 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,537 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,538 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,540 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,542 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,544 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,546 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,548 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T14:29:58,550 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,551 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,553 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,555 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,556 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,558 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,560 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,562 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,563 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,565 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,567 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,568 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,570 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,572 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,573 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,575 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,577 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,579 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,581 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,582 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T14:29:58,584 copying build/lib/lm_eval/tasks/afrobench/injongointent/injongointent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent 2026-05-11T14:29:58,587 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,588 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,590 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,591 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,593 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,595 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,597 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,599 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,600 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,602 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,604 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,605 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,607 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,609 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,611 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,612 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,614 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,616 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,618 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,619 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T14:29:58,621 copying build/lib/lm_eval/tasks/afrobench/injongointent/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent 2026-05-11T14:29:58,624 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,625 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,627 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,629 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,630 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,632 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,634 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,636 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,638 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,639 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,641 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,643 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,644 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,646 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,648 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,650 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,652 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,654 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,656 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,657 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T14:29:58,660 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,661 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,663 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,664 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,666 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,668 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,670 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,672 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,673 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,675 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,677 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,678 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,680 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,682 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,683 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,685 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,687 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,689 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,691 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,692 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T14:29:58,694 copying build/lib/lm_eval/tasks/afrobench/injongointent/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent 2026-05-11T14:29:58,697 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt 2026-05-11T14:29:58,699 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,700 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_nyn-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,702 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,704 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_ibo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,706 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_teo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,707 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,709 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_lgg-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,711 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_lug-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,713 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_swa-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,715 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,717 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-teo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,718 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-nyn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,720 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,722 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_ach-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,723 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lgg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,725 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T14:29:58,727 copying build/lib/lm_eval/tasks/afrobench/salt/salt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt 2026-05-11T14:29:58,729 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,730 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_nyn-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,732 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,734 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_ibo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,735 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_teo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,737 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,739 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_lgg-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,741 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_lug-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,742 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_swa-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,744 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,746 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-teo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,748 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-nyn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,749 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,751 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_ach-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,753 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lgg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,755 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T14:29:58,756 copying build/lib/lm_eval/tasks/afrobench/salt/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt 2026-05-11T14:29:58,759 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,760 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_nyn-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,762 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,764 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_ibo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,767 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_teo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,769 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,771 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_lgg-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,773 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_lug-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,776 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_swa-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,778 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,780 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-teo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,783 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-nyn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,785 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,787 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_ach-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,789 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lgg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,792 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T14:29:58,794 copying build/lib/lm_eval/tasks/afrobench/salt/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt 2026-05-11T14:29:58,798 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:29:58,799 copying build/lib/lm_eval/tasks/afrobench/masakhanews/masakhanews.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:29:58,802 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,803 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,806 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,808 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,811 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,813 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,815 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,817 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,819 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,821 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,823 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,825 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,828 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,830 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,832 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,834 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,836 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,839 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,841 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T14:29:58,844 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,845 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,847 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,850 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,852 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,854 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,857 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,859 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,861 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,863 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,865 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,867 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,869 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,870 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,872 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,874 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,876 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,878 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,880 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T14:29:58,882 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,883 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,885 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,887 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,889 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,890 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,892 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,894 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,896 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,898 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,900 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,902 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,904 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,906 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,908 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,910 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,911 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,913 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,915 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T14:29:58,916 copying build/lib/lm_eval/tasks/afrobench/masakhanews/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:29:58,919 copying build/lib/lm_eval/tasks/afrobench/masakhanews/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews 2026-05-11T14:29:58,922 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,923 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,925 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,927 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,929 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,931 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,933 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,934 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,936 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,938 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,940 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,942 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,944 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,946 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,948 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,950 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,952 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,954 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,956 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T14:29:58,958 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,960 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,962 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,964 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,966 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,968 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,970 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,972 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,974 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,976 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,978 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,980 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,982 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,984 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,986 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,988 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,990 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,991 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,993 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T14:29:58,995 copying build/lib/lm_eval/tasks/afrobench/afrobench-lite.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench 2026-05-11T14:29:58,998 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xquad 2026-05-11T14:29:58,999 copying build/lib/lm_eval/tasks/xquad/xquad_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,000 copying build/lib/lm_eval/tasks/xquad/xquad_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,002 copying build/lib/lm_eval/tasks/xquad/xquad_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,004 copying build/lib/lm_eval/tasks/xquad/xquad_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,005 copying build/lib/lm_eval/tasks/xquad/xquad_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,007 copying build/lib/lm_eval/tasks/xquad/xquad_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,009 copying build/lib/lm_eval/tasks/xquad/xquad_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,011 copying build/lib/lm_eval/tasks/xquad/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,012 copying build/lib/lm_eval/tasks/xquad/xquad_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,014 copying build/lib/lm_eval/tasks/xquad/xquad_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,016 copying build/lib/lm_eval/tasks/xquad/xquad_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,017 copying build/lib/lm_eval/tasks/xquad/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,019 copying build/lib/lm_eval/tasks/xquad/xquad_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,021 copying build/lib/lm_eval/tasks/xquad/xquad_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,022 copying build/lib/lm_eval/tasks/xquad/xquad_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T14:29:59,025 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,026 copying build/lib/lm_eval/tasks/spanish_bench/spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,028 copying build/lib/lm_eval/tasks/spanish_bench/openbookqa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,029 copying build/lib/lm_eval/tasks/spanish_bench/paws_es_spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,031 copying build/lib/lm_eval/tasks/spanish_bench/cocoteros_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,033 copying build/lib/lm_eval/tasks/spanish_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,036 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:59,036 copying build/lib/lm_eval/tasks/spanish_bench/phrases_es/_phrases_es_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:59,038 copying build/lib/lm_eval/tasks/spanish_bench/phrases_es/phrases_es-va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:59,040 copying build/lib/lm_eval/tasks/spanish_bench/phrases_es/phrases_va-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T14:29:59,042 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,043 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_eu-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,045 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,047 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_ca-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,048 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,050 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,051 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,053 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,055 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,057 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_pt-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,059 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,060 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,062 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,064 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_gl-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,066 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_fr-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,067 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_de-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,069 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_en-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,071 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,073 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_it-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,074 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/create_yamls_flores_es.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T14:29:59,077 copying build/lib/lm_eval/tasks/spanish_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,078 copying build/lib/lm_eval/tasks/spanish_bench/xlsum_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,080 copying build/lib/lm_eval/tasks/spanish_bench/copa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,082 copying build/lib/lm_eval/tasks/spanish_bench/escola.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,083 copying build/lib/lm_eval/tasks/spanish_bench/wnli_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,085 copying build/lib/lm_eval/tasks/spanish_bench/mgsm_direct_es_spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,087 copying build/lib/lm_eval/tasks/spanish_bench/xnli_es_spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T14:29:59,089 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2026-05-11T14:29:59,090 copying build/lib/lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,092 copying build/lib/lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,093 copying build/lib/lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,095 copying build/lib/lm_eval/tasks/paloma/paloma_wikitext_103.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,097 copying build/lib/lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,099 copying build/lib/lm_eval/tasks/paloma/paloma_ptb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,100 copying build/lib/lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,102 copying build/lib/lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,104 copying build/lib/lm_eval/tasks/paloma/paloma_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,105 copying build/lib/lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,107 copying build/lib/lm_eval/tasks/paloma/_paloma_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,109 copying build/lib/lm_eval/tasks/paloma/paloma_c4_100_domains.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,111 copying build/lib/lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,112 copying build/lib/lm_eval/tasks/paloma/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,114 copying build/lib/lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,116 copying build/lib/lm_eval/tasks/paloma/paloma_c4_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,118 copying build/lib/lm_eval/tasks/paloma/paloma_mc4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,119 copying build/lib/lm_eval/tasks/paloma/paloma_gab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,121 copying build/lib/lm_eval/tasks/paloma/paloma_redpajama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T14:29:59,124 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/med_text_classification 2026-05-11T14:29:59,125 copying build/lib/lm_eval/tasks/med_text_classification/med_text_classification_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_text_classification 2026-05-11T14:29:59,127 copying build/lib/lm_eval/tasks/med_text_classification/med_text_classification_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_text_classification 2026-05-11T14:29:59,128 copying build/lib/lm_eval/tasks/med_text_classification/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_text_classification 2026-05-11T14:29:59,131 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2026-05-11T14:29:59,132 copying build/lib/lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,133 copying build/lib/lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,135 copying build/lib/lm_eval/tasks/translation/wmt16_de-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,137 copying build/lib/lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,139 copying build/lib/lm_eval/tasks/translation/wmt_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,140 copying build/lib/lm_eval/tasks/translation/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,142 copying build/lib/lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,144 copying build/lib/lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,145 copying build/lib/lm_eval/tasks/translation/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,147 copying build/lib/lm_eval/tasks/translation/wmt16_en-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,149 copying build/lib/lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T14:29:59,152 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,152 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,154 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,156 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,158 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,159 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,161 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,163 copying build/lib/lm_eval/tasks/truthfulqa-multi/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,165 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,167 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,169 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,170 copying build/lib/lm_eval/tasks/truthfulqa-multi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,172 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,174 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,176 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,177 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,179 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,181 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,182 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,184 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T14:29:59,186 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cabbq 2026-05-11T14:29:59,187 copying build/lib/lm_eval/tasks/cabbq/cabbq_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,189 copying build/lib/lm_eval/tasks/cabbq/cabbq_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,190 copying build/lib/lm_eval/tasks/cabbq/cabbq_race_ethnicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,192 copying build/lib/lm_eval/tasks/cabbq/cabbq_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,194 copying build/lib/lm_eval/tasks/cabbq/cabbq_spanish_region.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,195 copying build/lib/lm_eval/tasks/cabbq/cabbq_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,197 copying build/lib/lm_eval/tasks/cabbq/cabbq_ses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,199 copying build/lib/lm_eval/tasks/cabbq/cabbq_disability_status.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,201 copying build/lib/lm_eval/tasks/cabbq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,202 copying build/lib/lm_eval/tasks/cabbq/cabbq_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,204 copying build/lib/lm_eval/tasks/cabbq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,206 copying build/lib/lm_eval/tasks/cabbq/cabbq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,208 copying build/lib/lm_eval/tasks/cabbq/cabbq_lgbtqia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,210 copying build/lib/lm_eval/tasks/cabbq/_cabbq_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T14:29:59,212 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/common_voice 2026-05-11T14:29:59,213 copying build/lib/lm_eval/tasks/common_voice/common_voice_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/common_voice 2026-05-11T14:29:59,215 copying build/lib/lm_eval/tasks/common_voice/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/common_voice 2026-05-11T14:29:59,217 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/groundcocoa 2026-05-11T14:29:59,218 copying build/lib/lm_eval/tasks/groundcocoa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/groundcocoa 2026-05-11T14:29:59,220 copying build/lib/lm_eval/tasks/groundcocoa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/groundcocoa 2026-05-11T14:29:59,222 copying build/lib/lm_eval/tasks/groundcocoa/groundcocoa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/groundcocoa 2026-05-11T14:29:59,224 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/fda 2026-05-11T14:29:59,225 copying build/lib/lm_eval/tasks/fda/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fda 2026-05-11T14:29:59,227 copying build/lib/lm_eval/tasks/fda/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fda 2026-05-11T14:29:59,229 copying build/lib/lm_eval/tasks/fda/fda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fda 2026-05-11T14:29:59,231 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2 2026-05-11T14:29:59,232 copying build/lib/lm_eval/tasks/squadv2/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squadv2 2026-05-11T14:29:59,234 copying build/lib/lm_eval/tasks/squadv2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squadv2 2026-05-11T14:29:59,236 copying build/lib/lm_eval/tasks/squadv2/squadv2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squadv2 2026-05-11T14:29:59,238 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aime 2026-05-11T14:29:59,239 copying build/lib/lm_eval/tasks/aime/aime.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T14:29:59,241 copying build/lib/lm_eval/tasks/aime/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T14:29:59,243 copying build/lib/lm_eval/tasks/aime/aime24.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T14:29:59,245 copying build/lib/lm_eval/tasks/aime/aime25.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T14:29:59,246 copying build/lib/lm_eval/tasks/aime/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T14:29:59,249 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2026-05-11T14:29:59,250 copying build/lib/lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,252 copying build/lib/lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,254 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,256 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,258 copying build/lib/lm_eval/tasks/french_bench/french_bench_xnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,259 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,261 copying build/lib/lm_eval/tasks/french_bench/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,263 copying build/lib/lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,265 copying build/lib/lm_eval/tasks/french_bench/preprocess_wikitext.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,266 copying build/lib/lm_eval/tasks/french_bench/french_bench_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,268 copying build/lib/lm_eval/tasks/french_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,270 copying build/lib/lm_eval/tasks/french_bench/french_bench_vocab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,272 copying build/lib/lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,274 copying build/lib/lm_eval/tasks/french_bench/french_bench_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,275 copying build/lib/lm_eval/tasks/french_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,277 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,279 copying build/lib/lm_eval/tasks/french_bench/french_bench_trivia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,280 copying build/lib/lm_eval/tasks/french_bench/french_bench_multifquad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,282 copying build/lib/lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,284 copying build/lib/lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,285 copying build/lib/lm_eval/tasks/french_bench/french_bench_reading_comp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,287 copying build/lib/lm_eval/tasks/french_bench/french_bench_boolqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T14:29:59,289 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench 2026-05-11T14:29:59,290 copying build/lib/lm_eval/tasks/eq_bench/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench 2026-05-11T14:29:59,293 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:29:59,294 copying build/lib/lm_eval/tasks/eq_bench/multilingual/eqbench_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:29:59,296 copying build/lib/lm_eval/tasks/eq_bench/multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:29:59,297 copying build/lib/lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench/multilingual 2026-05-11T14:29:59,299 copying build/lib/lm_eval/tasks/eq_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench 2026-05-11T14:29:59,301 copying build/lib/lm_eval/tasks/eq_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench 2026-05-11T14:29:59,303 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/esbbq 2026-05-11T14:29:59,305 copying build/lib/lm_eval/tasks/esbbq/esbbq_spanish_region.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,307 copying build/lib/lm_eval/tasks/esbbq/esbbq_race_ethnicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,308 copying build/lib/lm_eval/tasks/esbbq/esbbq_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,310 copying build/lib/lm_eval/tasks/esbbq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,312 copying build/lib/lm_eval/tasks/esbbq/esbbq_ses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,314 copying build/lib/lm_eval/tasks/esbbq/esbbq_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,315 copying build/lib/lm_eval/tasks/esbbq/esbbq_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,317 copying build/lib/lm_eval/tasks/esbbq/esbbq_disability_status.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,319 copying build/lib/lm_eval/tasks/esbbq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,321 copying build/lib/lm_eval/tasks/esbbq/esbbq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,323 copying build/lib/lm_eval/tasks/esbbq/esbbq_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,325 copying build/lib/lm_eval/tasks/esbbq/esbbq_lgbtqia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,326 copying build/lib/lm_eval/tasks/esbbq/_esbbq_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,328 copying build/lib/lm_eval/tasks/esbbq/esbbq_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T14:29:59,330 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc 2026-05-11T14:29:59,331 copying build/lib/lm_eval/tasks/arc/arc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T14:29:59,333 copying build/lib/lm_eval/tasks/arc/arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T14:29:59,334 copying build/lib/lm_eval/tasks/arc/arc_challenge_chat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T14:29:59,336 copying build/lib/lm_eval/tasks/arc/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T14:29:59,338 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts 2026-05-11T14:29:59,340 copying build/lib/lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/realtoxicityprompts 2026-05-11T14:29:59,341 copying build/lib/lm_eval/tasks/realtoxicityprompts/metric.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/realtoxicityprompts 2026-05-11T14:29:59,344 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bhs 2026-05-11T14:29:59,345 copying build/lib/lm_eval/tasks/bhs/bhs_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,347 copying build/lib/lm_eval/tasks/bhs/hindi-S_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,348 copying build/lib/lm_eval/tasks/bhs/basque-S-S_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,350 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_ni_A.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,352 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V1_V2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,353 copying build/lib/lm_eval/tasks/bhs/hindi-S_ne_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,355 copying build/lib/lm_eval/tasks/bhs/basque-S-IO_S_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,357 copying build/lib/lm_eval/tasks/bhs/basque-S-S_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,358 copying build/lib/lm_eval/tasks/bhs/basque-DO-S_IO_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,360 copying build/lib/lm_eval/tasks/bhs/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,362 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_ni_AN.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,364 copying build/lib/lm_eval/tasks/bhs/bhs_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,366 copying build/lib/lm_eval/tasks/bhs/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,367 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,369 copying build/lib/lm_eval/tasks/bhs/hindi-S_ne_PossPRN_PossN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,371 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_ni_A.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,372 copying build/lib/lm_eval/tasks/bhs/basque-S-S_IO_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,374 copying build/lib/lm_eval/tasks/bhs/bhs_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,376 copying build/lib/lm_eval/tasks/bhs/basque-DO-S_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,377 copying build/lib/lm_eval/tasks/bhs/basque-IO-IO_S_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,379 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_V_ni_AN.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,381 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,382 copying build/lib/lm_eval/tasks/bhs/hindi-S_ne_PossPRN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,384 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,385 copying build/lib/lm_eval/tasks/bhs/hindi-S_PossPRN_PossN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,387 copying build/lib/lm_eval/tasks/bhs/basque-IO-S_IO_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,389 copying build/lib/lm_eval/tasks/bhs/hindi-S_PossPRN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T14:29:59,391 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux 2026-05-11T14:29:59,393 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,394 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,396 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,398 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,400 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,402 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,404 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,406 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,408 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,410 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,412 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,413 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,415 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,417 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,419 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,420 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,422 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,424 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,425 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,427 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,429 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,430 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,432 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,434 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,436 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,437 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,439 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,441 copying build/lib/lm_eval/tasks/mmlu-redux/generative/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,443 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,444 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,446 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,448 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,450 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,456 copying build/lib/lm_eval/tasks/mmlu-redux/generative/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,458 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,460 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,462 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,464 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,466 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,468 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,469 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,471 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,473 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,474 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,476 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,478 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,479 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,481 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,483 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,485 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,487 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,489 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,491 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,493 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,494 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,496 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,498 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,500 copying build/lib/lm_eval/tasks/mmlu-redux/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,502 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,504 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,505 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T14:29:59,508 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze 2026-05-11T14:29:59,509 copying build/lib/lm_eval/tasks/lambada_cloze/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_cloze 2026-05-11T14:29:59,511 copying build/lib/lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_cloze 2026-05-11T14:29:59,513 copying build/lib/lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_cloze 2026-05-11T14:29:59,515 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs 2026-05-11T14:29:59,516 copying build/lib/lm_eval/tasks/webqs/webqs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/webqs 2026-05-11T14:29:59,518 copying build/lib/lm_eval/tasks/webqs/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/webqs 2026-05-11T14:29:59,520 copying build/lib/lm_eval/tasks/webqs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/webqs 2026-05-11T14:29:59,522 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks 2026-05-11T14:29:59,524 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:59,525 copying build/lib/lm_eval/tasks/benchmarks/flan/_held_in_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:59,527 copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:59,529 copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/flan 2026-05-11T14:29:59,531 copying build/lib/lm_eval/tasks/benchmarks/pythia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T14:29:59,533 copying build/lib/lm_eval/tasks/benchmarks/minerva_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T14:29:59,534 copying build/lib/lm_eval/tasks/benchmarks/t0_eval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T14:29:59,536 copying build/lib/lm_eval/tasks/benchmarks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T14:29:59,538 copying build/lib/lm_eval/tasks/benchmarks/openllm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T14:29:59,540 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T14:29:59,541 copying build/lib/lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/multimedqa 2026-05-11T14:29:59,543 copying build/lib/lm_eval/tasks/benchmarks/multimedqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/multimedqa 2026-05-11T14:29:59,545 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada 2026-05-11T14:29:59,546 copying build/lib/lm_eval/tasks/lambada/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada 2026-05-11T14:29:59,548 copying build/lib/lm_eval/tasks/lambada/lambada_openai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada 2026-05-11T14:29:59,550 copying build/lib/lm_eval/tasks/lambada/lambada_standard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada 2026-05-11T14:29:59,552 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2026-05-11T14:29:59,553 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T14:29:59,555 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T14:29:59,557 copying build/lib/lm_eval/tasks/gsm8k/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T14:29:59,559 copying build/lib/lm_eval/tasks/gsm8k/gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T14:29:59,560 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T14:29:59,562 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T14:29:59,565 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mts_dialog 2026-05-11T14:29:59,566 copying build/lib/lm_eval/tasks/mts_dialog/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T14:29:59,568 copying build/lib/lm_eval/tasks/mts_dialog/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T14:29:59,570 copying build/lib/lm_eval/tasks/mts_dialog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T14:29:59,573 copying build/lib/lm_eval/tasks/mts_dialog/mts_dialog_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T14:29:59,575 copying build/lib/lm_eval/tasks/mts_dialog/mts_dialog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T14:29:59,578 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darijammlu 2026-05-11T14:29:59,580 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,583 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,585 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,588 copying build/lib/lm_eval/tasks/darijammlu/_darijammlu_ar_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,590 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,593 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_political_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,595 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_natural_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,598 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,600 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,603 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,605 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,608 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,610 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,613 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,615 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_driving_test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,618 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,620 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,622 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_arabic_language_(grammar).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,625 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,628 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,630 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,633 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_philosophy_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,635 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,638 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,640 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,642 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,645 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,647 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_management_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,650 copying build/lib/lm_eval/tasks/darijammlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,653 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,655 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,658 copying build/lib/lm_eval/tasks/darijammlu/_darijammlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,660 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,663 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,665 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,667 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,669 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,671 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,673 copying build/lib/lm_eval/tasks/darijammlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,675 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,677 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,679 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_arabic_language_(general).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,681 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_arabic_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,683 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,685 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,688 copying build/lib/lm_eval/tasks/darijammlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,690 copying build/lib/lm_eval/tasks/darijammlu/_darijammlu_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,692 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,694 copying build/lib/lm_eval/tasks/darijammlu/_default_darijammlu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,696 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,698 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T14:29:59,700 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2026-05-11T14:29:59,701 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T14:29:59,704 copying build/lib/lm_eval/tasks/truthfulqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T14:29:59,706 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T14:29:59,708 copying build/lib/lm_eval/tasks/truthfulqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T14:29:59,710 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T14:29:59,713 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/meddialog 2026-05-11T14:29:59,714 copying build/lib/lm_eval/tasks/meddialog/meddialog_qsumm_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T14:29:59,716 copying build/lib/lm_eval/tasks/meddialog/meddialog_qsumm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T14:29:59,718 copying build/lib/lm_eval/tasks/meddialog/meddialog_raw_dialogues.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T14:29:59,720 copying build/lib/lm_eval/tasks/meddialog/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T14:29:59,722 copying build/lib/lm_eval/tasks/meddialog/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T14:29:59,724 copying build/lib/lm_eval/tasks/meddialog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T14:29:59,726 copying build/lib/lm_eval/tasks/meddialog/meddialog_raw_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T14:29:59,728 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/histoires_morales 2026-05-11T14:29:59,730 copying build/lib/lm_eval/tasks/histoires_morales/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/histoires_morales 2026-05-11T14:29:59,732 copying build/lib/lm_eval/tasks/histoires_morales/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/histoires_morales 2026-05-11T14:29:59,734 copying build/lib/lm_eval/tasks/histoires_morales/histoires_morales.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/histoires_morales 2026-05-11T14:29:59,737 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm_plus 2026-05-11T14:29:59,738 copying build/lib/lm_eval/tasks/gsm_plus/gsm_plus_mini.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm_plus 2026-05-11T14:29:59,740 copying build/lib/lm_eval/tasks/gsm_plus/gsm_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm_plus 2026-05-11T14:29:59,742 copying build/lib/lm_eval/tasks/gsm_plus/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm_plus 2026-05-11T14:29:59,745 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darijahellaswag 2026-05-11T14:29:59,746 copying build/lib/lm_eval/tasks/darijahellaswag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijahellaswag 2026-05-11T14:29:59,749 copying build/lib/lm_eval/tasks/darijahellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijahellaswag 2026-05-11T14:29:59,751 copying build/lib/lm_eval/tasks/darijahellaswag/darijahellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijahellaswag 2026-05-11T14:29:59,753 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,754 copying build/lib/lm_eval/tasks/kormedmcqa/_kormedmcqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,756 copying build/lib/lm_eval/tasks/kormedmcqa/dentist.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,758 copying build/lib/lm_eval/tasks/kormedmcqa/nurse.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,760 copying build/lib/lm_eval/tasks/kormedmcqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,762 copying build/lib/lm_eval/tasks/kormedmcqa/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,764 copying build/lib/lm_eval/tasks/kormedmcqa/pharm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,767 copying build/lib/lm_eval/tasks/kormedmcqa/doctor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T14:29:59,769 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2026-05-11T14:29:59,771 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,773 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,775 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,777 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,779 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,781 copying build/lib/lm_eval/tasks/arc_mt/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,783 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,785 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,787 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,789 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,791 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,793 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,795 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T14:29:59,798 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,799 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,801 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,803 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,805 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,807 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,809 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,811 copying build/lib/lm_eval/tasks/mmlu-pro-plus/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,813 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,815 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,817 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,819 copying build/lib/lm_eval/tasks/mmlu-pro-plus/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,821 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,823 copying build/lib/lm_eval/tasks/mmlu-pro-plus/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,825 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,827 copying build/lib/lm_eval/tasks/mmlu-pro-plus/_mmlu_pro_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,829 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,831 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,833 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T14:29:59,836 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273 2026-05-11T14:29:59,837 copying build/lib/lm_eval/tasks/wsc273/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wsc273 2026-05-11T14:29:59,839 copying build/lib/lm_eval/tasks/wsc273/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wsc273 2026-05-11T14:29:59,842 copying build/lib/lm_eval/tasks/wsc273/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wsc273 2026-05-11T14:29:59,845 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turblimp 2026-05-11T14:29:59,847 copying build/lib/lm_eval/tasks/turblimp/ellipsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,849 copying build/lib/lm_eval/tasks/turblimp/suspended_affixation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,851 copying build/lib/lm_eval/tasks/turblimp/scrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,853 copying build/lib/lm_eval/tasks/turblimp/nominalization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,855 copying build/lib/lm_eval/tasks/turblimp/quantifiers.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,858 copying build/lib/lm_eval/tasks/turblimp/relative_clauses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,860 copying build/lib/lm_eval/tasks/turblimp/island_effects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,862 copying build/lib/lm_eval/tasks/turblimp/binding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,864 copying build/lib/lm_eval/tasks/turblimp/irregular_forms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,866 copying build/lib/lm_eval/tasks/turblimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,868 copying build/lib/lm_eval/tasks/turblimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,870 copying build/lib/lm_eval/tasks/turblimp/turblimp_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,872 copying build/lib/lm_eval/tasks/turblimp/argument_structure_transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,874 copying build/lib/lm_eval/tasks/turblimp/subject_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,876 copying build/lib/lm_eval/tasks/turblimp/argument_structure_ditransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,878 copying build/lib/lm_eval/tasks/turblimp/determiners.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,880 copying build/lib/lm_eval/tasks/turblimp/anaphor_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,882 copying build/lib/lm_eval/tasks/turblimp/npi_licensing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,884 copying build/lib/lm_eval/tasks/turblimp/passives.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T14:29:59,886 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:59,888 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:59,890 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:59,892 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:59,894 copying build/lib/lm_eval/tasks/lambada_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:59,896 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:59,898 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T14:29:59,900 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score 2026-05-11T14:29:59,902 copying build/lib/lm_eval/tasks/score/score_robustness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T14:29:59,905 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,906 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,908 copying build/lib/lm_eval/tasks/score/agi_eval/score_option_order_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,910 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,912 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,913 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,915 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_aqua_rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,917 copying build/lib/lm_eval/tasks/score/agi_eval/score_prompt_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,919 copying build/lib/lm_eval/tasks/score/agi_eval/utils_agieval.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,921 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_templates.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,923 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_logiqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,925 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_logiqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,930 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lsat_rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,931 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,933 copying build/lib/lm_eval/tasks/score/agi_eval/score_non_greedy_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,935 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_aqua_rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,937 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,938 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,940 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lsat_rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,942 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,944 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,945 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,947 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,949 copying build/lib/lm_eval/tasks/score/agi_eval/score_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,951 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,953 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,954 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_logiqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,956 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_aqua_rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T14:29:59,958 copying build/lib/lm_eval/tasks/score/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T14:29:59,960 copying build/lib/lm_eval/tasks/score/non_greedy_summarizer.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T14:29:59,963 copying build/lib/lm_eval/tasks/score/NON_GREEDY.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T14:29:59,964 copying build/lib/lm_eval/tasks/score/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T14:29:59,968 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score/math 2026-05-11T14:29:59,969 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,971 copying build/lib/lm_eval/tasks/score/math/to_be_fixed_questions.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,973 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,974 copying build/lib/lm_eval/tasks/score/math/score_prompt_robustness_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,977 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,978 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,980 copying build/lib/lm_eval/tasks/score/math/utils_math.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,983 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,985 copying build/lib/lm_eval/tasks/score/math/score_non_greedy_robustness_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,988 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,989 copying build/lib/lm_eval/tasks/score/math/prompt_templates.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,991 copying build/lib/lm_eval/tasks/score/math/score_robustness_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,993 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,995 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,997 copying build/lib/lm_eval/tasks/score/math/math_grader.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:29:59,999 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:30:00,001 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:30:00,003 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:30:00,005 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:30:00,008 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:30:00,010 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T14:30:00,013 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score/mmlu_pro 2026-05-11T14:30:00,014 copying build/lib/lm_eval/tasks/score/mmlu_pro/utils_mmlu_pro.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T14:30:00,016 copying build/lib/lm_eval/tasks/score/mmlu_pro/prompt_templates.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T14:30:00,018 copying build/lib/lm_eval/tasks/score/mmlu_pro/score_non_greedy_robustness_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T14:30:00,020 copying build/lib/lm_eval/tasks/score/mmlu_pro/score_option_order_robustness_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T14:30:00,022 copying build/lib/lm_eval/tasks/score/mmlu_pro/score_prompt_robustness_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T14:30:00,024 copying build/lib/lm_eval/tasks/score/non_greedy.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T14:30:00,026 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu 2026-05-11T14:30:00,029 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,030 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,032 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,034 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,036 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,038 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,040 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,042 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,044 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,046 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,048 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,050 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,052 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,054 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,056 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,059 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,061 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,063 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,066 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,068 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,070 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,072 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,074 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,076 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,078 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,080 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,082 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,084 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,086 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,088 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,091 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,093 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,095 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,098 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,100 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,102 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,104 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,107 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,109 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,111 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,113 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,119 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,121 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,123 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,125 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,127 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,129 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,132 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,134 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,136 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,138 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,141 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,143 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,146 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,149 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,152 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,155 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,157 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,159 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,161 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,163 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T14:30:00,167 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,168 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,170 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,172 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,173 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,175 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,177 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,178 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,180 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,182 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,185 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,187 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,189 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,191 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,192 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,194 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,196 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,198 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,200 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,202 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,203 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,205 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,207 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,209 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,211 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,212 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,214 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,216 copying build/lib/lm_eval/tasks/mmlu/generative/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,218 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,220 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,221 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,223 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,225 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,226 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,228 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,230 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,232 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,234 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,235 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,237 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,239 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,241 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,242 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,244 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,246 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,248 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,249 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,251 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,253 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,255 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,257 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,258 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,260 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,262 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,264 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,265 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,267 copying build/lib/lm_eval/tasks/mmlu/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,269 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,270 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,272 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T14:30:00,274 copying build/lib/lm_eval/tasks/mmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu 2026-05-11T14:30:00,278 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,279 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,281 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,282 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,284 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,286 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,287 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,289 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,291 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,293 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,295 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,297 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,298 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,301 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,302 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,304 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,306 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,308 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,309 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,311 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,313 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,314 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,316 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,318 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,320 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,321 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,323 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,325 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,326 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,328 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,330 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,332 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,333 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,335 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,337 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,339 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,340 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,342 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,344 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,346 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,347 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,349 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,351 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,352 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,355 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,356 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,358 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,360 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,362 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,364 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,365 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,367 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,369 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,371 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,373 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,375 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,378 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,380 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,382 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,384 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,387 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T14:30:00,391 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,392 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,394 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,396 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,398 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,400 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,403 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,405 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,407 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,409 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,411 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,414 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,416 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,418 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,420 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,422 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,424 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,427 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,429 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,431 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,433 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,435 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,438 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,440 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,442 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,444 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,447 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,449 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,451 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,453 copying build/lib/lm_eval/tasks/mmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,455 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,457 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,459 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,461 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,463 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,465 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,468 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,470 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,472 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,473 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,475 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,477 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,478 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,481 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,490 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,491 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,493 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,495 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,497 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,498 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,500 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,502 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,504 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,505 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,507 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,508 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,510 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,512 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,513 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,515 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,517 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,519 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,521 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,522 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T14:30:00,525 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,526 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,528 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,530 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,532 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,533 copying build/lib/lm_eval/tasks/mmlu/continuation/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,535 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,537 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,539 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,541 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,542 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,544 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,546 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,548 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,549 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,551 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,553 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,554 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,556 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,558 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,560 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,562 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,563 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,565 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,567 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,569 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,571 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,572 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,574 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,576 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,577 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,579 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,581 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,582 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,584 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,586 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,588 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,589 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,591 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,593 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,594 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,596 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,598 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,600 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,601 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,603 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,605 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,607 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,608 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,610 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,612 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,614 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,616 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,618 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,620 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,621 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,623 copying build/lib/lm_eval/tasks/mmlu/continuation/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,625 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,627 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,628 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T14:30:00,631 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot 2026-05-11T14:30:00,633 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,634 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,636 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,638 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,639 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,641 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,642 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,644 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,646 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,648 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,649 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,652 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,654 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,656 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,658 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,660 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,661 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,663 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,665 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,668 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,670 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,672 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,674 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,676 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,678 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,679 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,681 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,683 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,685 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,686 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,688 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,690 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,692 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,694 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,695 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,697 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,699 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,701 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,703 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,704 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,706 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,708 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,710 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,711 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,714 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,715 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,717 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,719 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,721 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,723 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,724 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,726 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,728 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,730 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,731 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,733 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,735 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,737 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,738 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,740 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,742 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T14:30:00,745 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,746 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,748 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,750 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,752 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,753 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,755 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,757 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,759 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,760 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,762 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,764 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,766 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,768 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,770 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,771 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,773 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,775 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,777 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,778 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,780 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,782 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,784 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,785 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,787 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,789 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,790 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,792 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,794 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,796 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,797 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,799 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,801 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,803 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,805 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,807 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,809 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,811 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,813 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,815 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,816 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,818 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,820 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,822 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,824 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,825 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,827 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,829 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,831 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,833 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,834 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,836 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,838 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,840 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,842 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,844 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,846 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,848 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,850 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,852 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T14:30:00,854 copying build/lib/lm_eval/tasks/mmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu 2026-05-11T14:30:00,858 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,859 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,861 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,863 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,865 copying build/lib/lm_eval/tasks/evalita_llm/_re_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,866 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,868 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,870 copying build/lib/lm_eval/tasks/evalita_llm/_sa_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,872 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,873 copying build/lib/lm_eval/tasks/evalita_llm/_hs_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,875 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,877 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,879 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,880 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ls_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,882 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,884 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,886 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,888 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,889 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,891 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,893 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,895 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,897 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,898 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,900 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,902 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,904 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,906 copying build/lib/lm_eval/tasks/evalita_llm/_ls_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,908 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,909 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,911 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,913 copying build/lib/lm_eval/tasks/evalita_llm/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,915 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,917 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,919 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,920 copying build/lib/lm_eval/tasks/evalita_llm/_sa_template_v2_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,922 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,924 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_mc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,925 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_re_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,927 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,930 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,931 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ls_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,933 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,935 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,937 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ls_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,938 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,940 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,942 copying build/lib/lm_eval/tasks/evalita_llm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,943 copying build/lib/lm_eval/tasks/evalita_llm/_faq_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,945 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,947 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,949 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,951 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,953 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,955 copying build/lib/lm_eval/tasks/evalita_llm/_te_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,957 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,959 copying build/lib/lm_eval/tasks/evalita_llm/_sum_template_fp_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,960 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,962 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,964 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,966 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,968 copying build/lib/lm_eval/tasks/evalita_llm/_sum_template_fp-small_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,970 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,971 copying build/lib/lm_eval/tasks/evalita_llm/_at_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,973 copying build/lib/lm_eval/tasks/evalita_llm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,975 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,977 copying build/lib/lm_eval/tasks/evalita_llm/_sum_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,979 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,981 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,983 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,985 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,986 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,988 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,990 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,992 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,993 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,995 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_re_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,997 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:00,999 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,001 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,002 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,004 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,006 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,008 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,010 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,012 copying build/lib/lm_eval/tasks/evalita_llm/_wic_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,013 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_re_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,015 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,017 copying build/lib/lm_eval/tasks/evalita_llm/_ner_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,018 copying build/lib/lm_eval/tasks/evalita_llm/sum_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,020 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,022 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,023 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,025 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T14:30:01,027 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl 2026-05-11T14:30:01,029 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,030 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_statute_hallucination_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,032 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_statute_number_and_content_matching_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,034 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_offense_component_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,036 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,038 copying build/lib/lm_eval/tasks/kbl/knowledge/_kbl_knowledge_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,040 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_query_statute_matching_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,042 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_reasoning_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,044 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_legal_concept_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T14:30:01,046 copying build/lib/lm_eval/tasks/kbl/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl 2026-05-11T14:30:01,049 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam 2026-05-11T14:30:01,051 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,052 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,054 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,055 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,057 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,059 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,061 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,062 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2024.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,064 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,066 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,068 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,069 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,071 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,073 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,074 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T14:30:01,077 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,078 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2024.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,080 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,082 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,083 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,085 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,087 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,089 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,091 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,092 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,094 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,096 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,098 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,099 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,101 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T14:30:01,103 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,104 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2010.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,106 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,108 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,110 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,111 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,113 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,115 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,116 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,118 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,120 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,122 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,123 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,125 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,127 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,128 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2011.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T14:30:01,131 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,132 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2024.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,134 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,136 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,138 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,139 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,141 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,143 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,144 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,146 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,148 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,150 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,151 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,153 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,155 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2025.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,157 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T14:30:01,159 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/reasoning 2026-05-11T14:30:01,160 copying build/lib/lm_eval/tasks/kbl/reasoning/_kbl_reasoning_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T14:30:01,162 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_causal_reasoning_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T14:30:01,164 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_statement_consistency_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T14:30:01,166 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_q_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T14:30:01,168 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_p_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T14:30:01,171 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lingoly 2026-05-11T14:30:01,172 copying build/lib/lm_eval/tasks/lingoly/lingoly_nocontext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T14:30:01,174 copying build/lib/lm_eval/tasks/lingoly/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T14:30:01,175 copying build/lib/lm_eval/tasks/lingoly/lingoly_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T14:30:01,177 copying build/lib/lm_eval/tasks/lingoly/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T14:30:01,179 copying build/lib/lm_eval/tasks/lingoly/lingoly_context.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T14:30:01,181 copying build/lib/lm_eval/tasks/lingoly/script.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T14:30:01,183 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,184 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,186 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,188 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,190 copying build/lib/lm_eval/tasks/hendrycks_math/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,192 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,193 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,195 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,198 copying build/lib/lm_eval/tasks/hendrycks_math/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,200 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,202 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math500.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,203 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T14:30:01,206 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval 2026-05-11T14:30:01,208 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norec 2026-05-11T14:30:01,209 copying build/lib/lm_eval/tasks/noreval/norec/_norec_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec 2026-05-11T14:30:01,211 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:30:01,212 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:30:01,214 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:30:01,216 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:30:01,218 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:30:01,220 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T14:30:01,222 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:30:01,223 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:30:01,225 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:30:01,228 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:30:01,230 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:30:01,232 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T14:30:01,234 copying build/lib/lm_eval/tasks/noreval/norec/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec 2026-05-11T14:30:01,236 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,237 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,239 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,241 copying build/lib/lm_eval/tasks/noreval/norquad/_norquad_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,243 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,245 copying build/lib/lm_eval/tasks/noreval/norquad/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,247 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,249 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T14:30:01,251 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T14:30:01,253 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:30:01,254 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:30:01,256 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:30:01,258 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:30:01,260 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:30:01,262 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T14:30:01,264 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:30:01,266 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:30:01,268 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:30:01,270 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:30:01,272 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:30:01,273 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T14:30:01,275 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa 2026-05-11T14:30:01,277 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/_noropenbookqa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa 2026-05-11T14:30:01,279 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T14:30:01,280 copying build/lib/lm_eval/tasks/noreval/norrewrite-instruct/norrewrite_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T14:30:01,283 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba 2026-05-11T14:30:01,284 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:30:01,285 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:30:01,287 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:30:01,289 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:30:01,291 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T14:30:01,293 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:30:01,294 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:30:01,296 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:30:01,298 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:30:01,300 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T14:30:01,302 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:30:01,304 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:30:01,306 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:30:01,308 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:30:01,310 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T14:30:01,313 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:30:01,314 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:30:01,316 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:30:01,318 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:30:01,319 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T14:30:01,321 copying build/lib/lm_eval/tasks/noreval/tatoeba/_tatoeba_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba 2026-05-11T14:30:01,323 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T14:30:01,324 copying build/lib/lm_eval/tasks/noreval/norsummarize-instruct/norsummarize_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T14:30:01,326 copying build/lib/lm_eval/tasks/noreval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval 2026-05-11T14:30:01,329 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsumm 2026-05-11T14:30:01,330 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:30:01,332 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:30:01,333 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:30:01,335 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:30:01,337 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:30:01,339 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:30:01,341 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T14:30:01,343 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:30:01,344 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:30:01,347 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:30:01,349 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:30:01,351 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:30:01,353 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:30:01,355 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T14:30:01,357 copying build/lib/lm_eval/tasks/noreval/norsumm/_norsumm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm 2026-05-11T14:30:01,359 copying build/lib/lm_eval/tasks/noreval/norsumm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm 2026-05-11T14:30:01,361 copying build/lib/lm_eval/tasks/noreval/noreval.jpg -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval 2026-05-11T14:30:01,375 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norcommonsenseqa 2026-05-11T14:30:01,377 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:30:01,378 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:30:01,380 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:30:01,383 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:30:01,385 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:30:01,387 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T14:30:01,389 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:30:01,390 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:30:01,393 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:30:01,395 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:30:01,396 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:30:01,399 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T14:30:01,400 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/_norcommonsenseqa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa 2026-05-11T14:30:01,403 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,404 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,406 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,407 copying build/lib/lm_eval/tasks/noreval/ask_gec/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,409 copying build/lib/lm_eval/tasks/noreval/ask_gec/_ask_gec_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,411 copying build/lib/lm_eval/tasks/noreval/ask_gec/errant.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,413 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,415 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,417 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T14:30:01,419 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norbelebele 2026-05-11T14:30:01,420 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T14:30:01,422 copying build/lib/lm_eval/tasks/noreval/norbelebele/_norbelebele_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T14:30:01,424 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T14:30:01,426 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T14:30:01,428 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T14:30:01,430 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T14:30:01,432 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/ncb 2026-05-11T14:30:01,433 copying build/lib/lm_eval/tasks/noreval/ncb/ncb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ncb 2026-05-11T14:30:01,435 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noridiom 2026-05-11T14:30:01,437 copying build/lib/lm_eval/tasks/noreval/noridiom/_noridiom_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom 2026-05-11T14:30:01,439 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:30:01,440 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:30:01,442 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:30:01,444 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:30:01,446 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:30:01,448 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T14:30:01,450 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:30:01,451 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:30:01,453 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:30:01,455 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:30:01,457 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:30:01,459 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T14:30:01,460 copying build/lib/lm_eval/tasks/noreval/noridiom/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom 2026-05-11T14:30:01,463 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa 2026-05-11T14:30:01,464 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice 2026-05-11T14:30:01,466 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:30:01,468 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:30:01,470 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:30:01,471 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:30:01,473 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:30:01,475 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:30:01,477 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T14:30:01,479 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/_nortruthfulqa_mc_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice 2026-05-11T14:30:01,482 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:30:01,483 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:30:01,485 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:30:01,486 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:30:01,488 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:30:01,490 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:30:01,492 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T14:30:01,494 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T14:30:01,496 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:30:01,497 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:30:01,499 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:30:01,501 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:30:01,503 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:30:01,505 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T14:30:01,507 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:30:01,508 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:30:01,510 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:30:01,512 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:30:01,514 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:30:01,516 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T14:30:01,519 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T14:30:01,521 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/_nortruthfulqa_gen_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T14:30:01,523 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nrk_quiz_qa 2026-05-11T14:30:01,524 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/_nrk_quiz_qa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa 2026-05-11T14:30:01,527 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:30:01,528 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:30:01,530 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:30:01,532 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:30:01,534 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:30:01,535 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:30:01,537 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T14:30:01,540 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:30:01,541 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:30:01,543 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:30:01,545 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:30:01,547 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:30:01,549 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:30:01,551 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T14:30:01,553 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr 2026-05-11T14:30:01,556 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,557 copying build/lib/lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,559 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,562 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,564 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,566 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,568 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,570 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,572 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,573 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,575 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,577 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,579 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,581 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,583 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,585 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,587 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,589 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,591 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,593 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,595 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,597 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,599 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,601 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,603 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,605 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,607 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,609 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,611 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,613 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,615 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,616 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,618 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,620 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,622 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,624 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,625 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,627 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,629 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,631 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,633 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,635 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,637 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,639 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,640 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,642 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,644 copying build/lib/lm_eval/tasks/mmlusr/question_only/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,646 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,648 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,650 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,651 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,653 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,655 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,657 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,658 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,660 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,662 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,664 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,665 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,667 copying build/lib/lm_eval/tasks/mmlusr/question_only/_question_only.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,669 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T14:30:01,671 copying build/lib/lm_eval/tasks/mmlusr/config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr 2026-05-11T14:30:01,673 copying build/lib/lm_eval/tasks/mmlusr/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr 2026-05-11T14:30:01,677 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,678 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,680 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,682 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,683 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,685 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,687 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,689 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,691 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,692 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,694 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,696 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,698 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,700 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,701 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,703 copying build/lib/lm_eval/tasks/mmlusr/answer_only/_mmlusr_a_yml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,705 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,707 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,708 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,710 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,712 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,714 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,716 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,717 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,719 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,721 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,723 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,725 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,727 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,728 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,730 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,732 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,734 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,736 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,738 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,740 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,741 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,743 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,745 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,747 copying build/lib/lm_eval/tasks/mmlusr/answer_only/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,749 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,751 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,752 copying build/lib/lm_eval/tasks/mmlusr/answer_only/_answer_only.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,754 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,756 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,758 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,760 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,761 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,763 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,765 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,767 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,769 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,771 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,773 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,775 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,777 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,779 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,780 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,782 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,784 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,786 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T14:30:01,789 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,790 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,792 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,794 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,795 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,797 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,799 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,801 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,803 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,805 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,807 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,809 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,811 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,813 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,816 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,818 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,820 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,822 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,824 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,825 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,828 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,830 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,832 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,833 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,835 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,837 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,839 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,841 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,843 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/_question_and_answer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,844 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,846 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,848 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,850 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,852 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,854 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,856 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,858 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,860 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,862 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,864 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,865 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,867 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,869 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,871 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,873 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,874 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,876 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/_mmlusr_qna_yml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,878 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,880 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,882 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,884 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,886 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,888 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,890 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,892 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,894 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,896 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,898 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,900 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,902 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,904 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T14:30:01,907 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/egyhellaswag 2026-05-11T14:30:01,908 copying build/lib/lm_eval/tasks/egyhellaswag/egyhellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egyhellaswag 2026-05-11T14:30:01,910 copying build/lib/lm_eval/tasks/egyhellaswag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egyhellaswag 2026-05-11T14:30:01,912 copying build/lib/lm_eval/tasks/egyhellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egyhellaswag 2026-05-11T14:30:01,914 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext 2026-05-11T14:30:01,915 copying build/lib/lm_eval/tasks/wikitext/wikitext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wikitext 2026-05-11T14:30:01,917 copying build/lib/lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wikitext 2026-05-11T14:30:01,919 copying build/lib/lm_eval/tasks/wikitext/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wikitext 2026-05-11T14:30:01,922 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,923 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,925 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,927 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,929 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,931 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,933 copying build/lib/lm_eval/tasks/mmlu_pro/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,935 copying build/lib/lm_eval/tasks/mmlu_pro/_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,937 copying build/lib/lm_eval/tasks/mmlu_pro/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,939 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,941 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,943 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,945 copying build/lib/lm_eval/tasks/mmlu_pro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,947 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,949 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,951 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,953 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,955 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,957 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T14:30:01,960 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2026-05-11T14:30:01,961 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,963 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,965 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,967 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,969 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,970 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,972 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,974 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,976 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,978 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,980 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,982 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,983 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,985 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,987 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,989 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,991 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,992 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,994 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,996 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:01,998 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,000 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,002 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,004 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,005 copying build/lib/lm_eval/tasks/eus_exams/configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,007 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,009 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,011 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,012 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,014 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,016 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,017 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,019 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,021 copying build/lib/lm_eval/tasks/eus_exams/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,023 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,025 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,026 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,029 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,030 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,032 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,034 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,036 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,038 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,040 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,042 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,044 copying build/lib/lm_eval/tasks/eus_exams/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,046 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,048 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,050 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,051 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,053 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,055 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,057 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,059 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,061 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,062 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,064 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,066 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,068 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,070 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,072 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,074 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,076 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,078 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,080 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,082 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,084 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,086 copying build/lib/lm_eval/tasks/eus_exams/eus_exams -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T14:30:02,091 creating build/bdist.linux-armv7l/wheel/lm_eval/filters 2026-05-11T14:30:02,093 copying build/lib/lm_eval/filters/custom.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T14:30:02,095 copying build/lib/lm_eval/filters/transformation.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T14:30:02,097 copying build/lib/lm_eval/filters/extraction.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T14:30:02,099 copying build/lib/lm_eval/filters/selection.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T14:30:02,101 copying build/lib/lm_eval/filters/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T14:30:02,103 copying build/lib/lm_eval/filters/decontamination.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T14:30:02,105 copying build/lib/lm_eval/__main__.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T14:30:02,106 copying build/lib/lm_eval/evaluator.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T14:30:02,110 creating build/bdist.linux-armv7l/wheel/lm_eval/config 2026-05-11T14:30:02,111 copying build/lib/lm_eval/config/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T14:30:02,113 copying build/lib/lm_eval/config/evaluate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T14:30:02,116 copying build/lib/lm_eval/config/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T14:30:02,117 copying build/lib/lm_eval/config/group.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T14:30:02,120 copying build/lib/lm_eval/result_schema.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T14:30:02,122 copying build/lib/lm_eval/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T14:30:02,124 copying build/lib/lm_eval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T14:30:02,126 copying build/lib/lm_eval/evaluator_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T14:30:02,130 creating build/bdist.linux-armv7l/wheel/lm_eval/models 2026-05-11T14:30:02,131 copying build/lib/lm_eval/models/openai_completions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,133 copying build/lib/lm_eval/models/megatron_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,137 copying build/lib/lm_eval/models/sglang_causallms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,139 copying build/lib/lm_eval/models/huggingface.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,143 copying build/lib/lm_eval/models/textsynth.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,145 copying build/lib/lm_eval/models/utils_hf.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,147 copying build/lib/lm_eval/models/optimum_habana.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,149 copying build/lib/lm_eval/models/hf_audiolm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,152 copying build/lib/lm_eval/models/hf_vlms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,154 copying build/lib/lm_eval/models/neuron_optimum.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,157 copying build/lib/lm_eval/models/litellm_llms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,159 copying build/lib/lm_eval/models/optimum_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,161 copying build/lib/lm_eval/models/anthropic_llms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,163 copying build/lib/lm_eval/models/sglang_generate_API.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,165 copying build/lib/lm_eval/models/api_models.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,168 copying build/lib/lm_eval/models/vllm_vlms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,170 copying build/lib/lm_eval/models/ibm_watsonx_ai.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,173 copying build/lib/lm_eval/models/mistral3.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,175 copying build/lib/lm_eval/models/mamba_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,177 copying build/lib/lm_eval/models/vllm_causallms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,180 copying build/lib/lm_eval/models/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,182 copying build/lib/lm_eval/models/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,184 copying build/lib/lm_eval/models/gguf.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,186 copying build/lib/lm_eval/models/dummy.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,188 copying build/lib/lm_eval/models/nemo_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,191 copying build/lib/lm_eval/models/winml.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,193 copying build/lib/lm_eval/models/hf_steered.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,195 copying build/lib/lm_eval/models/trtllm_causallms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,198 copying build/lib/lm_eval/models/optimum_ipex.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T14:30:02,200 copying build/lib/lm_eval/defaults.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T14:30:02,202 creating build/bdist.linux-armv7l/wheel/lm_eval/loggers 2026-05-11T14:30:02,203 copying build/lib/lm_eval/loggers/evaluation_tracker.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T14:30:02,206 copying build/lib/lm_eval/loggers/trackio_logger.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T14:30:02,208 copying build/lib/lm_eval/loggers/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T14:30:02,210 copying build/lib/lm_eval/loggers/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T14:30:02,212 copying build/lib/lm_eval/loggers/wandb_logger.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T14:30:02,215 creating build/bdist.linux-armv7l/wheel/lm_eval/_cli 2026-05-11T14:30:02,216 copying build/lib/lm_eval/_cli/validate.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T14:30:02,219 copying build/lib/lm_eval/_cli/ls.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T14:30:02,221 copying build/lib/lm_eval/_cli/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T14:30:02,222 copying build/lib/lm_eval/_cli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T14:30:02,224 copying build/lib/lm_eval/_cli/harness.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T14:30:02,226 copying build/lib/lm_eval/_cli/run.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T14:30:02,229 copying build/lib/lm_eval/_cli/subcommand.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T14:30:02,234 running install_egg_info 2026-05-11T14:30:02,240 Copying lm_eval.egg-info to build/bdist.linux-armv7l/wheel/./lm_eval-0.4.12-py3.11.egg-info 2026-05-11T14:30:02,267 running install_scripts 2026-05-11T14:30:02,279 creating build/bdist.linux-armv7l/wheel/lm_eval-0.4.12.dist-info/WHEEL 2026-05-11T14:30:02,282 creating '/tmp/pip-wheel-ek8ksrwg/.tmp-1iwfi0lt/lm_eval-0.4.12-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2026-05-11T14:30:02,284 adding 'lm_eval/__init__.py' 2026-05-11T14:30:02,286 adding 'lm_eval/__main__.py' 2026-05-11T14:30:02,287 adding 'lm_eval/defaults.py' 2026-05-11T14:30:02,291 adding 'lm_eval/evaluator.py' 2026-05-11T14:30:02,294 adding 'lm_eval/evaluator_utils.py' 2026-05-11T14:30:02,296 adding 'lm_eval/result_schema.py' 2026-05-11T14:30:02,300 adding 'lm_eval/utils.py' 2026-05-11T14:30:02,302 adding 'lm_eval/_cli/__init__.py' 2026-05-11T14:30:02,303 adding 'lm_eval/_cli/harness.py' 2026-05-11T14:30:02,305 adding 'lm_eval/_cli/ls.py' 2026-05-11T14:30:02,307 adding 'lm_eval/_cli/run.py' 2026-05-11T14:30:02,308 adding 'lm_eval/_cli/subcommand.py' 2026-05-11T14:30:02,310 adding 'lm_eval/_cli/utils.py' 2026-05-11T14:30:02,311 adding 'lm_eval/_cli/validate.py' 2026-05-11T14:30:02,313 adding 'lm_eval/api/__init__.py' 2026-05-11T14:30:02,314 adding 'lm_eval/api/filter.py' 2026-05-11T14:30:02,317 adding 'lm_eval/api/group.py' 2026-05-11T14:30:02,318 adding 'lm_eval/api/instance.py' 2026-05-11T14:30:02,320 adding 'lm_eval/api/metrics.py' 2026-05-11T14:30:02,323 adding 'lm_eval/api/model.py' 2026-05-11T14:30:02,326 adding 'lm_eval/api/registry.py' 2026-05-11T14:30:02,328 adding 'lm_eval/api/samplers.py' 2026-05-11T14:30:02,336 adding 'lm_eval/api/task.py' 2026-05-11T14:30:02,338 adding 'lm_eval/api/utils.py' 2026-05-11T14:30:02,339 adding 'lm_eval/caching/__init__.py' 2026-05-11T14:30:02,340 adding 'lm_eval/caching/cache.py' 2026-05-11T14:30:02,342 adding 'lm_eval/config/__init__.py' 2026-05-11T14:30:02,344 adding 'lm_eval/config/evaluate_config.py' 2026-05-11T14:30:02,346 adding 'lm_eval/config/group.py' 2026-05-11T14:30:02,347 adding 'lm_eval/config/task.py' 2026-05-11T14:30:02,349 adding 'lm_eval/decontamination/__init__.py' 2026-05-11T14:30:02,351 adding 'lm_eval/decontamination/archiver.py' 2026-05-11T14:30:02,352 adding 'lm_eval/decontamination/decontaminate.py' 2026-05-11T14:30:02,354 adding 'lm_eval/decontamination/janitor.py' 2026-05-11T14:30:02,356 adding 'lm_eval/filters/__init__.py' 2026-05-11T14:30:02,357 adding 'lm_eval/filters/custom.py' 2026-05-11T14:30:02,359 adding 'lm_eval/filters/decontamination.py' 2026-05-11T14:30:02,360 adding 'lm_eval/filters/extraction.py' 2026-05-11T14:30:02,362 adding 'lm_eval/filters/selection.py' 2026-05-11T14:30:02,363 adding 'lm_eval/filters/transformation.py' 2026-05-11T14:30:02,365 adding 'lm_eval/loggers/__init__.py' 2026-05-11T14:30:02,368 adding 'lm_eval/loggers/evaluation_tracker.py' 2026-05-11T14:30:02,370 adding 'lm_eval/loggers/trackio_logger.py' 2026-05-11T14:30:02,371 adding 'lm_eval/loggers/utils.py' 2026-05-11T14:30:02,374 adding 'lm_eval/loggers/wandb_logger.py' 2026-05-11T14:30:02,376 adding 'lm_eval/models/__init__.py' 2026-05-11T14:30:02,378 adding 'lm_eval/models/anthropic_llms.py' 2026-05-11T14:30:02,382 adding 'lm_eval/models/api_models.py' 2026-05-11T14:30:02,383 adding 'lm_eval/models/dummy.py' 2026-05-11T14:30:02,385 adding 'lm_eval/models/gguf.py' 2026-05-11T14:30:02,387 adding 'lm_eval/models/hf_audiolm.py' 2026-05-11T14:30:02,389 adding 'lm_eval/models/hf_steered.py' 2026-05-11T14:30:02,393 adding 'lm_eval/models/hf_vlms.py' 2026-05-11T14:30:02,402 adding 'lm_eval/models/huggingface.py' 2026-05-11T14:30:02,405 adding 'lm_eval/models/ibm_watsonx_ai.py' 2026-05-11T14:30:02,407 adding 'lm_eval/models/litellm_llms.py' 2026-05-11T14:30:02,409 adding 'lm_eval/models/mamba_lm.py' 2026-05-11T14:30:02,415 adding 'lm_eval/models/megatron_lm.py' 2026-05-11T14:30:02,416 adding 'lm_eval/models/mistral3.py' 2026-05-11T14:30:02,419 adding 'lm_eval/models/nemo_lm.py' 2026-05-11T14:30:02,423 adding 'lm_eval/models/neuron_optimum.py' 2026-05-11T14:30:02,425 adding 'lm_eval/models/openai_completions.py' 2026-05-11T14:30:02,426 adding 'lm_eval/models/optimum_habana.py' 2026-05-11T14:30:02,428 adding 'lm_eval/models/optimum_ipex.py' 2026-05-11T14:30:02,429 adding 'lm_eval/models/optimum_lm.py' 2026-05-11T14:30:02,432 adding 'lm_eval/models/sglang_causallms.py' 2026-05-11T14:30:02,434 adding 'lm_eval/models/sglang_generate_API.py' 2026-05-11T14:30:02,435 adding 'lm_eval/models/textsynth.py' 2026-05-11T14:30:02,438 adding 'lm_eval/models/trtllm_causallms.py' 2026-05-11T14:30:02,443 adding 'lm_eval/models/utils.py' 2026-05-11T14:30:02,445 adding 'lm_eval/models/utils_hf.py' 2026-05-11T14:30:02,449 adding 'lm_eval/models/vllm_causallms.py' 2026-05-11T14:30:02,451 adding 'lm_eval/models/vllm_vlms.py' 2026-05-11T14:30:02,455 adding 'lm_eval/models/winml.py' 2026-05-11T14:30:02,457 adding 'lm_eval/prompts/__init__.py' 2026-05-11T14:30:02,467 adding 'lm_eval/tasks/README.md' 2026-05-11T14:30:02,469 adding 'lm_eval/tasks/__init__.py' 2026-05-11T14:30:02,471 adding 'lm_eval/tasks/_factory.py' 2026-05-11T14:30:02,473 adding 'lm_eval/tasks/_index.py' 2026-05-11T14:30:02,474 adding 'lm_eval/tasks/_yaml_loader.py' 2026-05-11T14:30:02,476 adding 'lm_eval/tasks/manager.py' 2026-05-11T14:30:02,478 adding 'lm_eval/tasks/aclue/README.md' 2026-05-11T14:30:02,479 adding 'lm_eval/tasks/aclue/_aclue.yaml' 2026-05-11T14:30:02,480 adding 'lm_eval/tasks/aclue/_default_template_yaml' 2026-05-11T14:30:02,482 adding 'lm_eval/tasks/aclue/_generate_configs.py' 2026-05-11T14:30:02,483 adding 'lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml' 2026-05-11T14:30:02,484 adding 'lm_eval/tasks/aclue/aclue_ancient_literature.yaml' 2026-05-11T14:30:02,485 adding 'lm_eval/tasks/aclue/aclue_ancient_medical.yaml' 2026-05-11T14:30:02,486 adding 'lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml' 2026-05-11T14:30:02,487 adding 'lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml' 2026-05-11T14:30:02,488 adding 'lm_eval/tasks/aclue/aclue_couplet_prediction.yaml' 2026-05-11T14:30:02,489 adding 'lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml' 2026-05-11T14:30:02,490 adding 'lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml' 2026-05-11T14:30:02,491 adding 'lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml' 2026-05-11T14:30:02,493 adding 'lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml' 2026-05-11T14:30:02,494 adding 'lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml' 2026-05-11T14:30:02,495 adding 'lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml' 2026-05-11T14:30:02,496 adding 'lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml' 2026-05-11T14:30:02,497 adding 'lm_eval/tasks/aclue/aclue_reading_comprehension.yaml' 2026-05-11T14:30:02,498 adding 'lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml' 2026-05-11T14:30:02,500 adding 'lm_eval/tasks/acpbench/README.md' 2026-05-11T14:30:02,502 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/_boolq_cot_2shot_yaml' 2026-05-11T14:30:02,503 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/act_reach.yaml' 2026-05-11T14:30:02,505 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/app.yaml' 2026-05-11T14:30:02,506 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/just.yaml' 2026-05-11T14:30:02,508 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/land.yaml' 2026-05-11T14:30:02,509 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/prog.yaml' 2026-05-11T14:30:02,510 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/reach.yaml' 2026-05-11T14:30:02,512 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/val.yaml' 2026-05-11T14:30:02,514 adding 'lm_eval/tasks/acpbench/gen_2shot/_gen_yaml_2shot' 2026-05-11T14:30:02,515 adding 'lm_eval/tasks/acpbench/gen_2shot/acp_grammar.lark' 2026-05-11T14:30:02,519 adding 'lm_eval/tasks/acpbench/gen_2shot/acp_utils.py' 2026-05-11T14:30:02,521 adding 'lm_eval/tasks/acpbench/gen_2shot/act_reach.yaml' 2026-05-11T14:30:02,522 adding 'lm_eval/tasks/acpbench/gen_2shot/app.yaml' 2026-05-11T14:30:02,523 adding 'lm_eval/tasks/acpbench/gen_2shot/just.yaml' 2026-05-11T14:30:02,525 adding 'lm_eval/tasks/acpbench/gen_2shot/land.yaml' 2026-05-11T14:30:02,526 adding 'lm_eval/tasks/acpbench/gen_2shot/next_act.yaml' 2026-05-11T14:30:02,528 adding 'lm_eval/tasks/acpbench/gen_2shot/prog.yaml' 2026-05-11T14:30:02,529 adding 'lm_eval/tasks/acpbench/gen_2shot/reach.yaml' 2026-05-11T14:30:02,531 adding 'lm_eval/tasks/acpbench/gen_2shot/val.yaml' 2026-05-11T14:30:02,532 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/_gen_yaml_2shot' 2026-05-11T14:30:02,534 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_grammar.lark' 2026-05-11T14:30:02,538 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_utils.py' 2026-05-11T14:30:02,540 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/act_reach.yaml' 2026-05-11T14:30:02,542 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/app.yaml' 2026-05-11T14:30:02,544 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/just.yaml' 2026-05-11T14:30:02,546 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/land.yaml' 2026-05-11T14:30:02,548 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/next_act.yaml' 2026-05-11T14:30:02,549 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/prog.yaml' 2026-05-11T14:30:02,551 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/reach.yaml' 2026-05-11T14:30:02,553 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/val.yaml' 2026-05-11T14:30:02,555 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/_mcq_cot_2shot_yaml' 2026-05-11T14:30:02,556 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/act_reach.yaml' 2026-05-11T14:30:02,558 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/app.yaml' 2026-05-11T14:30:02,559 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/just.yaml' 2026-05-11T14:30:02,560 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/land.yaml' 2026-05-11T14:30:02,562 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/prog.yaml' 2026-05-11T14:30:02,563 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/reach.yaml' 2026-05-11T14:30:02,564 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/val.yaml' 2026-05-11T14:30:02,566 adding 'lm_eval/tasks/aexams/README.md' 2026-05-11T14:30:02,568 adding 'lm_eval/tasks/aexams/_aexams.yaml' 2026-05-11T14:30:02,569 adding 'lm_eval/tasks/aexams/_default_template_yaml' 2026-05-11T14:30:02,570 adding 'lm_eval/tasks/aexams/aexams_Biology.yaml' 2026-05-11T14:30:02,571 adding 'lm_eval/tasks/aexams/aexams_IslamicStudies.yaml' 2026-05-11T14:30:02,572 adding 'lm_eval/tasks/aexams/aexams_Physics.yaml' 2026-05-11T14:30:02,573 adding 'lm_eval/tasks/aexams/aexams_Science.yaml' 2026-05-11T14:30:02,575 adding 'lm_eval/tasks/aexams/aexams_Social.yaml' 2026-05-11T14:30:02,577 adding 'lm_eval/tasks/afrimgsm/README.md' 2026-05-11T14:30:02,579 adding 'lm_eval/tasks/afrimgsm/gen_utils.py' 2026-05-11T14:30:02,580 adding 'lm_eval/tasks/afrimgsm/gen_yaml.sh' 2026-05-11T14:30:02,581 adding 'lm_eval/tasks/afrimgsm/run.sh' 2026-05-11T14:30:02,583 adding 'lm_eval/tasks/afrimgsm/utils.py' 2026-05-11T14:30:02,584 adding 'lm_eval/tasks/afrimgsm/direct/afrimgsm.yaml' 2026-05-11T14:30:02,586 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_amh.yaml' 2026-05-11T14:30:02,587 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_eng.yaml' 2026-05-11T14:30:02,588 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ewe.yaml' 2026-05-11T14:30:02,589 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_fra.yaml' 2026-05-11T14:30:02,591 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_hau.yaml' 2026-05-11T14:30:02,592 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ibo.yaml' 2026-05-11T14:30:02,593 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_kin.yaml' 2026-05-11T14:30:02,594 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lin.yaml' 2026-05-11T14:30:02,595 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lug.yaml' 2026-05-11T14:30:02,596 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_orm.yaml' 2026-05-11T14:30:02,597 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sna.yaml' 2026-05-11T14:30:02,598 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sot.yaml' 2026-05-11T14:30:02,600 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_swa.yaml' 2026-05-11T14:30:02,601 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_twi.yaml' 2026-05-11T14:30:02,602 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_vai.yaml' 2026-05-11T14:30:02,603 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_wol.yaml' 2026-05-11T14:30:02,604 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_xho.yaml' 2026-05-11T14:30:02,605 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yaml' 2026-05-11T14:30:02,607 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yor.yaml' 2026-05-11T14:30:02,608 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_zul.yaml' 2026-05-11T14:30:02,610 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_amh.yaml' 2026-05-11T14:30:02,611 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_eng.yaml' 2026-05-11T14:30:02,613 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ewe.yaml' 2026-05-11T14:30:02,614 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_fra.yaml' 2026-05-11T14:30:02,615 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_hau.yaml' 2026-05-11T14:30:02,616 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ibo.yaml' 2026-05-11T14:30:02,617 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_kin.yaml' 2026-05-11T14:30:02,618 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lin.yaml' 2026-05-11T14:30:02,619 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lug.yaml' 2026-05-11T14:30:02,621 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_orm.yaml' 2026-05-11T14:30:02,622 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sna.yaml' 2026-05-11T14:30:02,623 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sot.yaml' 2026-05-11T14:30:02,624 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_swa.yaml' 2026-05-11T14:30:02,625 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_twi.yaml' 2026-05-11T14:30:02,626 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_vai.yaml' 2026-05-11T14:30:02,627 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_wol.yaml' 2026-05-11T14:30:02,629 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_xho.yaml' 2026-05-11T14:30:02,630 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yaml' 2026-05-11T14:30:02,631 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yor.yaml' 2026-05-11T14:30:02,632 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_zul.yaml' 2026-05-11T14:30:02,634 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_amh.yaml' 2026-05-11T14:30:02,635 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_eng.yaml' 2026-05-11T14:30:02,636 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ewe.yaml' 2026-05-11T14:30:02,637 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_fra.yaml' 2026-05-11T14:30:02,638 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_hau.yaml' 2026-05-11T14:30:02,639 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ibo.yaml' 2026-05-11T14:30:02,640 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_kin.yaml' 2026-05-11T14:30:02,641 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lin.yaml' 2026-05-11T14:30:02,643 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lug.yaml' 2026-05-11T14:30:02,644 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_orm.yaml' 2026-05-11T14:30:02,645 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sna.yaml' 2026-05-11T14:30:02,646 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sot.yaml' 2026-05-11T14:30:02,647 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_swa.yaml' 2026-05-11T14:30:02,648 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_twi.yaml' 2026-05-11T14:30:02,650 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_vai.yaml' 2026-05-11T14:30:02,651 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_wol.yaml' 2026-05-11T14:30:02,652 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_xho.yaml' 2026-05-11T14:30:02,653 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yaml' 2026-05-11T14:30:02,654 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yor.yaml' 2026-05-11T14:30:02,655 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_zul.yaml' 2026-05-11T14:30:02,657 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_amh.yaml' 2026-05-11T14:30:02,658 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_eng.yaml' 2026-05-11T14:30:02,660 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ewe.yaml' 2026-05-11T14:30:02,661 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_fra.yaml' 2026-05-11T14:30:02,662 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_hau.yaml' 2026-05-11T14:30:02,663 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ibo.yaml' 2026-05-11T14:30:02,665 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_kin.yaml' 2026-05-11T14:30:02,666 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lin.yaml' 2026-05-11T14:30:02,667 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lug.yaml' 2026-05-11T14:30:02,668 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_orm.yaml' 2026-05-11T14:30:02,669 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sna.yaml' 2026-05-11T14:30:02,670 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sot.yaml' 2026-05-11T14:30:02,671 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_swa.yaml' 2026-05-11T14:30:02,672 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_twi.yaml' 2026-05-11T14:30:02,673 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_vai.yaml' 2026-05-11T14:30:02,674 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_wol.yaml' 2026-05-11T14:30:02,676 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_xho.yaml' 2026-05-11T14:30:02,677 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yaml' 2026-05-11T14:30:02,678 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yor.yaml' 2026-05-11T14:30:02,679 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_zul.yaml' 2026-05-11T14:30:02,681 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_amh.yaml' 2026-05-11T14:30:02,682 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_eng.yaml' 2026-05-11T14:30:02,683 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ewe.yaml' 2026-05-11T14:30:02,685 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_fra.yaml' 2026-05-11T14:30:02,686 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_hau.yaml' 2026-05-11T14:30:02,687 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ibo.yaml' 2026-05-11T14:30:02,688 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_kin.yaml' 2026-05-11T14:30:02,689 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lin.yaml' 2026-05-11T14:30:02,690 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lug.yaml' 2026-05-11T14:30:02,692 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_orm.yaml' 2026-05-11T14:30:02,693 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sna.yaml' 2026-05-11T14:30:02,694 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sot.yaml' 2026-05-11T14:30:02,695 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_swa.yaml' 2026-05-11T14:30:02,696 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_twi.yaml' 2026-05-11T14:30:02,698 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_vai.yaml' 2026-05-11T14:30:02,699 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_wol.yaml' 2026-05-11T14:30:02,700 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_xho.yaml' 2026-05-11T14:30:02,701 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yaml' 2026-05-11T14:30:02,702 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yor.yaml' 2026-05-11T14:30:02,703 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_zul.yaml' 2026-05-11T14:30:02,705 adding 'lm_eval/tasks/afrimgsm/direct_cot/afrimgsm_cot.yaml' 2026-05-11T14:30:02,707 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_amh.yaml' 2026-05-11T14:30:02,708 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_eng.yaml' 2026-05-11T14:30:02,710 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ewe.yaml' 2026-05-11T14:30:02,711 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_fra.yaml' 2026-05-11T14:30:02,712 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_hau.yaml' 2026-05-11T14:30:02,713 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ibo.yaml' 2026-05-11T14:30:02,714 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_kin.yaml' 2026-05-11T14:30:02,715 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lin.yaml' 2026-05-11T14:30:02,716 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lug.yaml' 2026-05-11T14:30:02,718 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_orm.yaml' 2026-05-11T14:30:02,719 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sna.yaml' 2026-05-11T14:30:02,720 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sot.yaml' 2026-05-11T14:30:02,721 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_swa.yaml' 2026-05-11T14:30:02,722 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_twi.yaml' 2026-05-11T14:30:02,723 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_vai.yaml' 2026-05-11T14:30:02,724 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_wol.yaml' 2026-05-11T14:30:02,725 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_xho.yaml' 2026-05-11T14:30:02,726 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yaml' 2026-05-11T14:30:02,728 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yor.yaml' 2026-05-11T14:30:02,729 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_zul.yaml' 2026-05-11T14:30:02,730 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_amh.yaml' 2026-05-11T14:30:02,732 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_eng.yaml' 2026-05-11T14:30:02,733 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ewe.yaml' 2026-05-11T14:30:02,734 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_fra.yaml' 2026-05-11T14:30:02,735 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_hau.yaml' 2026-05-11T14:30:02,736 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ibo.yaml' 2026-05-11T14:30:02,737 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_kin.yaml' 2026-05-11T14:30:02,738 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lin.yaml' 2026-05-11T14:30:02,739 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lug.yaml' 2026-05-11T14:30:02,740 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_orm.yaml' 2026-05-11T14:30:02,742 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sna.yaml' 2026-05-11T14:30:02,743 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sot.yaml' 2026-05-11T14:30:02,744 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_swa.yaml' 2026-05-11T14:30:02,745 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_twi.yaml' 2026-05-11T14:30:02,746 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_vai.yaml' 2026-05-11T14:30:02,747 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_wol.yaml' 2026-05-11T14:30:02,749 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_xho.yaml' 2026-05-11T14:30:02,750 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yaml' 2026-05-11T14:30:02,751 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yor.yaml' 2026-05-11T14:30:02,752 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_zul.yaml' 2026-05-11T14:30:02,754 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_amh.yaml' 2026-05-11T14:30:02,755 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_eng.yaml' 2026-05-11T14:30:02,756 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ewe.yaml' 2026-05-11T14:30:02,758 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_fra.yaml' 2026-05-11T14:30:02,759 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_hau.yaml' 2026-05-11T14:30:02,760 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ibo.yaml' 2026-05-11T14:30:02,761 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_kin.yaml' 2026-05-11T14:30:02,762 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lin.yaml' 2026-05-11T14:30:02,763 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lug.yaml' 2026-05-11T14:30:02,764 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_orm.yaml' 2026-05-11T14:30:02,766 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sna.yaml' 2026-05-11T14:30:02,767 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sot.yaml' 2026-05-11T14:30:02,768 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_swa.yaml' 2026-05-11T14:30:02,769 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_twi.yaml' 2026-05-11T14:30:02,770 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_vai.yaml' 2026-05-11T14:30:02,771 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_wol.yaml' 2026-05-11T14:30:02,772 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_xho.yaml' 2026-05-11T14:30:02,773 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yaml' 2026-05-11T14:30:02,774 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yor.yaml' 2026-05-11T14:30:02,775 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_zul.yaml' 2026-05-11T14:30:02,777 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_amh.yaml' 2026-05-11T14:30:02,778 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_eng.yaml' 2026-05-11T14:30:02,779 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ewe.yaml' 2026-05-11T14:30:02,780 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_fra.yaml' 2026-05-11T14:30:02,781 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_hau.yaml' 2026-05-11T14:30:02,782 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ibo.yaml' 2026-05-11T14:30:02,783 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_kin.yaml' 2026-05-11T14:30:02,785 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lin.yaml' 2026-05-11T14:30:02,786 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lug.yaml' 2026-05-11T14:30:02,787 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_orm.yaml' 2026-05-11T14:30:02,788 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sna.yaml' 2026-05-11T14:30:02,789 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sot.yaml' 2026-05-11T14:30:02,790 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_swa.yaml' 2026-05-11T14:30:02,792 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_twi.yaml' 2026-05-11T14:30:02,793 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_vai.yaml' 2026-05-11T14:30:02,794 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_wol.yaml' 2026-05-11T14:30:02,795 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_xho.yaml' 2026-05-11T14:30:02,796 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yaml' 2026-05-11T14:30:02,798 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yor.yaml' 2026-05-11T14:30:02,799 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_zul.yaml' 2026-05-11T14:30:02,801 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_amh.yaml' 2026-05-11T14:30:02,802 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_eng.yaml' 2026-05-11T14:30:02,803 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ewe.yaml' 2026-05-11T14:30:02,804 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_fra.yaml' 2026-05-11T14:30:02,805 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_hau.yaml' 2026-05-11T14:30:02,807 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ibo.yaml' 2026-05-11T14:30:02,808 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_kin.yaml' 2026-05-11T14:30:02,809 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lin.yaml' 2026-05-11T14:30:02,810 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lug.yaml' 2026-05-11T14:30:02,812 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_orm.yaml' 2026-05-11T14:30:02,813 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sna.yaml' 2026-05-11T14:30:02,814 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sot.yaml' 2026-05-11T14:30:02,815 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_swa.yaml' 2026-05-11T14:30:02,816 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_twi.yaml' 2026-05-11T14:30:02,817 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_vai.yaml' 2026-05-11T14:30:02,819 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_wol.yaml' 2026-05-11T14:30:02,820 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_xho.yaml' 2026-05-11T14:30:02,821 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yaml' 2026-05-11T14:30:02,823 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yor.yaml' 2026-05-11T14:30:02,824 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_zul.yaml' 2026-05-11T14:30:02,826 adding 'lm_eval/tasks/afrimgsm/translate/afrimgsm_tt.yaml' 2026-05-11T14:30:02,827 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_amh.yaml' 2026-05-11T14:30:02,829 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ewe.yaml' 2026-05-11T14:30:02,830 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_fra.yaml' 2026-05-11T14:30:02,831 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_hau.yaml' 2026-05-11T14:30:02,832 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ibo.yaml' 2026-05-11T14:30:02,833 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_kin.yaml' 2026-05-11T14:30:02,834 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lin.yaml' 2026-05-11T14:30:02,836 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lug.yaml' 2026-05-11T14:30:02,837 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_orm.yaml' 2026-05-11T14:30:02,838 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sna.yaml' 2026-05-11T14:30:02,839 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sot.yaml' 2026-05-11T14:30:02,840 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_swa.yaml' 2026-05-11T14:30:02,841 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_twi.yaml' 2026-05-11T14:30:02,842 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_wol.yaml' 2026-05-11T14:30:02,844 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_xho.yaml' 2026-05-11T14:30:02,845 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yaml' 2026-05-11T14:30:02,846 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yor.yaml' 2026-05-11T14:30:02,847 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_zul.yaml' 2026-05-11T14:30:02,849 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_amh.yaml' 2026-05-11T14:30:02,850 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ewe.yaml' 2026-05-11T14:30:02,851 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_fra.yaml' 2026-05-11T14:30:02,853 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_hau.yaml' 2026-05-11T14:30:02,854 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ibo.yaml' 2026-05-11T14:30:02,855 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_kin.yaml' 2026-05-11T14:30:02,856 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lin.yaml' 2026-05-11T14:30:02,857 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lug.yaml' 2026-05-11T14:30:02,858 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_orm.yaml' 2026-05-11T14:30:02,859 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sna.yaml' 2026-05-11T14:30:02,861 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sot.yaml' 2026-05-11T14:30:02,862 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_swa.yaml' 2026-05-11T14:30:02,863 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_twi.yaml' 2026-05-11T14:30:02,864 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_wol.yaml' 2026-05-11T14:30:02,865 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_xho.yaml' 2026-05-11T14:30:02,866 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yaml' 2026-05-11T14:30:02,868 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yor.yaml' 2026-05-11T14:30:02,869 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_zul.yaml' 2026-05-11T14:30:02,870 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_amh.yaml' 2026-05-11T14:30:02,871 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ewe.yaml' 2026-05-11T14:30:02,872 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_fra.yaml' 2026-05-11T14:30:02,874 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_hau.yaml' 2026-05-11T14:30:02,875 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ibo.yaml' 2026-05-11T14:30:02,876 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_kin.yaml' 2026-05-11T14:30:02,877 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lin.yaml' 2026-05-11T14:30:02,878 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lug.yaml' 2026-05-11T14:30:02,879 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_orm.yaml' 2026-05-11T14:30:02,880 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sna.yaml' 2026-05-11T14:30:02,881 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sot.yaml' 2026-05-11T14:30:02,883 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_swa.yaml' 2026-05-11T14:30:02,884 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_twi.yaml' 2026-05-11T14:30:02,885 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_wol.yaml' 2026-05-11T14:30:02,886 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_xho.yaml' 2026-05-11T14:30:02,887 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yaml' 2026-05-11T14:30:02,889 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yor.yaml' 2026-05-11T14:30:02,890 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_zul.yaml' 2026-05-11T14:30:02,892 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_amh.yaml' 2026-05-11T14:30:02,893 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ewe.yaml' 2026-05-11T14:30:02,894 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_fra.yaml' 2026-05-11T14:30:02,895 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_hau.yaml' 2026-05-11T14:30:02,896 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ibo.yaml' 2026-05-11T14:30:02,898 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_kin.yaml' 2026-05-11T14:30:02,899 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lin.yaml' 2026-05-11T14:30:02,900 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lug.yaml' 2026-05-11T14:30:02,901 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_orm.yaml' 2026-05-11T14:30:02,902 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sna.yaml' 2026-05-11T14:30:02,903 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sot.yaml' 2026-05-11T14:30:02,905 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_swa.yaml' 2026-05-11T14:30:02,906 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_twi.yaml' 2026-05-11T14:30:02,907 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_wol.yaml' 2026-05-11T14:30:02,908 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_xho.yaml' 2026-05-11T14:30:02,910 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yaml' 2026-05-11T14:30:02,911 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yor.yaml' 2026-05-11T14:30:02,912 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_zul.yaml' 2026-05-11T14:30:02,914 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_amh.yaml' 2026-05-11T14:30:02,915 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ewe.yaml' 2026-05-11T14:30:02,917 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_fra.yaml' 2026-05-11T14:30:02,918 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_hau.yaml' 2026-05-11T14:30:02,919 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ibo.yaml' 2026-05-11T14:30:02,920 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_kin.yaml' 2026-05-11T14:30:02,921 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lin.yaml' 2026-05-11T14:30:02,922 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lug.yaml' 2026-05-11T14:30:02,923 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_orm.yaml' 2026-05-11T14:30:02,925 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sna.yaml' 2026-05-11T14:30:02,926 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sot.yaml' 2026-05-11T14:30:02,927 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_swa.yaml' 2026-05-11T14:30:02,928 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_twi.yaml' 2026-05-11T14:30:02,929 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_wol.yaml' 2026-05-11T14:30:02,930 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_xho.yaml' 2026-05-11T14:30:02,931 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yaml' 2026-05-11T14:30:02,933 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yor.yaml' 2026-05-11T14:30:02,934 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_zul.yaml' 2026-05-11T14:30:02,936 adding 'lm_eval/tasks/afrimgsm/translate_cot/afrimgsm_tt_cot.yaml' 2026-05-11T14:30:02,938 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_amh.yaml' 2026-05-11T14:30:02,939 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T14:30:02,940 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_fra.yaml' 2026-05-11T14:30:02,942 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_hau.yaml' 2026-05-11T14:30:02,943 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T14:30:02,944 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_kin.yaml' 2026-05-11T14:30:02,945 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lin.yaml' 2026-05-11T14:30:02,946 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lug.yaml' 2026-05-11T14:30:02,947 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_orm.yaml' 2026-05-11T14:30:02,949 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sna.yaml' 2026-05-11T14:30:02,950 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sot.yaml' 2026-05-11T14:30:02,951 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_swa.yaml' 2026-05-11T14:30:02,952 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_twi.yaml' 2026-05-11T14:30:02,953 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_vai.yaml' 2026-05-11T14:30:02,954 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_wol.yaml' 2026-05-11T14:30:02,955 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_xho.yaml' 2026-05-11T14:30:02,957 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yaml' 2026-05-11T14:30:02,958 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yor.yaml' 2026-05-11T14:30:02,959 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_zul.yaml' 2026-05-11T14:30:02,960 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_amh.yaml' 2026-05-11T14:30:02,961 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T14:30:02,962 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_fra.yaml' 2026-05-11T14:30:02,963 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_hau.yaml' 2026-05-11T14:30:02,964 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T14:30:02,966 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_kin.yaml' 2026-05-11T14:30:02,967 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lin.yaml' 2026-05-11T14:30:02,968 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lug.yaml' 2026-05-11T14:30:02,969 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_orm.yaml' 2026-05-11T14:30:02,970 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sna.yaml' 2026-05-11T14:30:02,971 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sot.yaml' 2026-05-11T14:30:02,972 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_swa.yaml' 2026-05-11T14:30:02,973 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_twi.yaml' 2026-05-11T14:30:02,974 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_vai.yaml' 2026-05-11T14:30:02,975 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_wol.yaml' 2026-05-11T14:30:02,977 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_xho.yaml' 2026-05-11T14:30:02,978 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yaml' 2026-05-11T14:30:02,979 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yor.yaml' 2026-05-11T14:30:02,980 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_zul.yaml' 2026-05-11T14:30:02,982 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_amh.yaml' 2026-05-11T14:30:02,983 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T14:30:02,991 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_fra.yaml' 2026-05-11T14:30:02,992 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_hau.yaml' 2026-05-11T14:30:02,993 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T14:30:02,994 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_kin.yaml' 2026-05-11T14:30:02,995 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lin.yaml' 2026-05-11T14:30:02,997 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lug.yaml' 2026-05-11T14:30:02,998 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_orm.yaml' 2026-05-11T14:30:02,999 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sna.yaml' 2026-05-11T14:30:03,000 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sot.yaml' 2026-05-11T14:30:03,001 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_swa.yaml' 2026-05-11T14:30:03,002 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_twi.yaml' 2026-05-11T14:30:03,004 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_vai.yaml' 2026-05-11T14:30:03,005 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_wol.yaml' 2026-05-11T14:30:03,006 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_xho.yaml' 2026-05-11T14:30:03,007 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yaml' 2026-05-11T14:30:03,008 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yor.yaml' 2026-05-11T14:30:03,009 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_zul.yaml' 2026-05-11T14:30:03,011 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_amh.yaml' 2026-05-11T14:30:03,012 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T14:30:03,013 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_fra.yaml' 2026-05-11T14:30:03,015 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_hau.yaml' 2026-05-11T14:30:03,016 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T14:30:03,017 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_kin.yaml' 2026-05-11T14:30:03,018 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lin.yaml' 2026-05-11T14:30:03,020 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lug.yaml' 2026-05-11T14:30:03,021 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_orm.yaml' 2026-05-11T14:30:03,022 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sna.yaml' 2026-05-11T14:30:03,023 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sot.yaml' 2026-05-11T14:30:03,025 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_swa.yaml' 2026-05-11T14:30:03,027 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_twi.yaml' 2026-05-11T14:30:03,028 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_vai.yaml' 2026-05-11T14:30:03,029 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_wol.yaml' 2026-05-11T14:30:03,030 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_xho.yaml' 2026-05-11T14:30:03,032 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yaml' 2026-05-11T14:30:03,033 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yor.yaml' 2026-05-11T14:30:03,034 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_zul.yaml' 2026-05-11T14:30:03,036 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_amh.yaml' 2026-05-11T14:30:03,037 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T14:30:03,039 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_fra.yaml' 2026-05-11T14:30:03,040 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_hau.yaml' 2026-05-11T14:30:03,041 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T14:30:03,042 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_kin.yaml' 2026-05-11T14:30:03,044 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lin.yaml' 2026-05-11T14:30:03,045 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lug.yaml' 2026-05-11T14:30:03,047 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_orm.yaml' 2026-05-11T14:30:03,048 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sna.yaml' 2026-05-11T14:30:03,049 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sot.yaml' 2026-05-11T14:30:03,050 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_swa.yaml' 2026-05-11T14:30:03,051 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_twi.yaml' 2026-05-11T14:30:03,053 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_vai.yaml' 2026-05-11T14:30:03,054 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_wol.yaml' 2026-05-11T14:30:03,055 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_xho.yaml' 2026-05-11T14:30:03,056 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yaml' 2026-05-11T14:30:03,058 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yor.yaml' 2026-05-11T14:30:03,059 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_zul.yaml' 2026-05-11T14:30:03,061 adding 'lm_eval/tasks/afrimmlu/README.md' 2026-05-11T14:30:03,062 adding 'lm_eval/tasks/afrimmlu/fewshot.sh' 2026-05-11T14:30:03,063 adding 'lm_eval/tasks/afrimmlu/gen_utils.py' 2026-05-11T14:30:03,064 adding 'lm_eval/tasks/afrimmlu/utils.py' 2026-05-11T14:30:03,066 adding 'lm_eval/tasks/afrimmlu/direct/afrimmlu.yaml' 2026-05-11T14:30:03,068 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct' 2026-05-11T14:30:03,069 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_amh.yaml' 2026-05-11T14:30:03,070 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_eng.yaml' 2026-05-11T14:30:03,071 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ewe.yaml' 2026-05-11T14:30:03,073 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_fra.yaml' 2026-05-11T14:30:03,074 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_hau.yaml' 2026-05-11T14:30:03,075 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ibo.yaml' 2026-05-11T14:30:03,076 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_kin.yaml' 2026-05-11T14:30:03,077 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lin.yaml' 2026-05-11T14:30:03,078 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lug.yaml' 2026-05-11T14:30:03,079 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_orm.yaml' 2026-05-11T14:30:03,080 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sna.yaml' 2026-05-11T14:30:03,082 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sot.yaml' 2026-05-11T14:30:03,083 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_swa.yaml' 2026-05-11T14:30:03,084 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_twi.yaml' 2026-05-11T14:30:03,085 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_wol.yaml' 2026-05-11T14:30:03,086 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_xho.yaml' 2026-05-11T14:30:03,087 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_yor.yaml' 2026-05-11T14:30:03,088 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_zul.yaml' 2026-05-11T14:30:03,090 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/utils.py' 2026-05-11T14:30:03,092 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct' 2026-05-11T14:30:03,093 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_amh.yaml' 2026-05-11T14:30:03,094 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_eng.yaml' 2026-05-11T14:30:03,095 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ewe.yaml' 2026-05-11T14:30:03,096 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_fra.yaml' 2026-05-11T14:30:03,098 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_hau.yaml' 2026-05-11T14:30:03,099 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ibo.yaml' 2026-05-11T14:30:03,100 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_kin.yaml' 2026-05-11T14:30:03,101 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lin.yaml' 2026-05-11T14:30:03,102 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lug.yaml' 2026-05-11T14:30:03,103 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_orm.yaml' 2026-05-11T14:30:03,105 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sna.yaml' 2026-05-11T14:30:03,106 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sot.yaml' 2026-05-11T14:30:03,107 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_swa.yaml' 2026-05-11T14:30:03,108 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_twi.yaml' 2026-05-11T14:30:03,109 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_wol.yaml' 2026-05-11T14:30:03,110 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_xho.yaml' 2026-05-11T14:30:03,111 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_yor.yaml' 2026-05-11T14:30:03,112 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_zul.yaml' 2026-05-11T14:30:03,113 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/utils.py' 2026-05-11T14:30:03,115 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct' 2026-05-11T14:30:03,116 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_amh.yaml' 2026-05-11T14:30:03,117 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_eng.yaml' 2026-05-11T14:30:03,119 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ewe.yaml' 2026-05-11T14:30:03,120 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_fra.yaml' 2026-05-11T14:30:03,121 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_hau.yaml' 2026-05-11T14:30:03,122 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ibo.yaml' 2026-05-11T14:30:03,123 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_kin.yaml' 2026-05-11T14:30:03,124 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lin.yaml' 2026-05-11T14:30:03,125 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lug.yaml' 2026-05-11T14:30:03,126 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_orm.yaml' 2026-05-11T14:30:03,128 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sna.yaml' 2026-05-11T14:30:03,129 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sot.yaml' 2026-05-11T14:30:03,130 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_swa.yaml' 2026-05-11T14:30:03,131 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_twi.yaml' 2026-05-11T14:30:03,132 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_wol.yaml' 2026-05-11T14:30:03,134 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_xho.yaml' 2026-05-11T14:30:03,135 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_yor.yaml' 2026-05-11T14:30:03,136 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_zul.yaml' 2026-05-11T14:30:03,137 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/utils.py' 2026-05-11T14:30:03,139 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct' 2026-05-11T14:30:03,141 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_amh.yaml' 2026-05-11T14:30:03,142 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_eng.yaml' 2026-05-11T14:30:03,143 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ewe.yaml' 2026-05-11T14:30:03,144 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_fra.yaml' 2026-05-11T14:30:03,145 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_hau.yaml' 2026-05-11T14:30:03,146 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ibo.yaml' 2026-05-11T14:30:03,148 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_kin.yaml' 2026-05-11T14:30:03,149 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lin.yaml' 2026-05-11T14:30:03,150 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lug.yaml' 2026-05-11T14:30:03,151 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_orm.yaml' 2026-05-11T14:30:03,152 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sna.yaml' 2026-05-11T14:30:03,153 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sot.yaml' 2026-05-11T14:30:03,155 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_swa.yaml' 2026-05-11T14:30:03,156 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_twi.yaml' 2026-05-11T14:30:03,157 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_wol.yaml' 2026-05-11T14:30:03,158 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_xho.yaml' 2026-05-11T14:30:03,159 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_yor.yaml' 2026-05-11T14:30:03,160 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_zul.yaml' 2026-05-11T14:30:03,161 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/utils.py' 2026-05-11T14:30:03,164 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct' 2026-05-11T14:30:03,165 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_amh.yaml' 2026-05-11T14:30:03,166 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_eng.yaml' 2026-05-11T14:30:03,167 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ewe.yaml' 2026-05-11T14:30:03,168 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_fra.yaml' 2026-05-11T14:30:03,170 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_hau.yaml' 2026-05-11T14:30:03,171 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ibo.yaml' 2026-05-11T14:30:03,172 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_kin.yaml' 2026-05-11T14:30:03,173 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lin.yaml' 2026-05-11T14:30:03,174 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lug.yaml' 2026-05-11T14:30:03,175 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_orm.yaml' 2026-05-11T14:30:03,176 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sna.yaml' 2026-05-11T14:30:03,178 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sot.yaml' 2026-05-11T14:30:03,179 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_swa.yaml' 2026-05-11T14:30:03,180 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_twi.yaml' 2026-05-11T14:30:03,181 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_wol.yaml' 2026-05-11T14:30:03,182 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_xho.yaml' 2026-05-11T14:30:03,183 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_yor.yaml' 2026-05-11T14:30:03,185 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_zul.yaml' 2026-05-11T14:30:03,186 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/utils.py' 2026-05-11T14:30:03,188 adding 'lm_eval/tasks/afrimmlu/translate/afrimmlu_tt.yaml' 2026-05-11T14:30:03,190 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate' 2026-05-11T14:30:03,191 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_amh.yaml' 2026-05-11T14:30:03,192 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ewe.yaml' 2026-05-11T14:30:03,193 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_fra.yaml' 2026-05-11T14:30:03,194 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_hau.yaml' 2026-05-11T14:30:03,195 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ibo.yaml' 2026-05-11T14:30:03,197 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_kin.yaml' 2026-05-11T14:30:03,198 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lin.yaml' 2026-05-11T14:30:03,199 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lug.yaml' 2026-05-11T14:30:03,200 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_orm.yaml' 2026-05-11T14:30:03,201 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sna.yaml' 2026-05-11T14:30:03,202 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sot.yaml' 2026-05-11T14:30:03,204 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_swa.yaml' 2026-05-11T14:30:03,205 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_twi.yaml' 2026-05-11T14:30:03,206 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_wol.yaml' 2026-05-11T14:30:03,207 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_xho.yaml' 2026-05-11T14:30:03,208 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_yor.yaml' 2026-05-11T14:30:03,209 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_zul.yaml' 2026-05-11T14:30:03,210 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/utils.py' 2026-05-11T14:30:03,212 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate' 2026-05-11T14:30:03,213 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_amh.yaml' 2026-05-11T14:30:03,214 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ewe.yaml' 2026-05-11T14:30:03,215 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_fra.yaml' 2026-05-11T14:30:03,216 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_hau.yaml' 2026-05-11T14:30:03,217 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ibo.yaml' 2026-05-11T14:30:03,218 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_kin.yaml' 2026-05-11T14:30:03,219 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lin.yaml' 2026-05-11T14:30:03,220 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lug.yaml' 2026-05-11T14:30:03,221 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_orm.yaml' 2026-05-11T14:30:03,223 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sna.yaml' 2026-05-11T14:30:03,224 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sot.yaml' 2026-05-11T14:30:03,225 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_swa.yaml' 2026-05-11T14:30:03,226 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_twi.yaml' 2026-05-11T14:30:03,228 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_wol.yaml' 2026-05-11T14:30:03,229 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_xho.yaml' 2026-05-11T14:30:03,230 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_yor.yaml' 2026-05-11T14:30:03,232 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_zul.yaml' 2026-05-11T14:30:03,234 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/utils.py' 2026-05-11T14:30:03,238 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate' 2026-05-11T14:30:03,240 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_amh.yaml' 2026-05-11T14:30:03,241 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ewe.yaml' 2026-05-11T14:30:03,242 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_fra.yaml' 2026-05-11T14:30:03,243 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_hau.yaml' 2026-05-11T14:30:03,244 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ibo.yaml' 2026-05-11T14:30:03,246 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_kin.yaml' 2026-05-11T14:30:03,247 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lin.yaml' 2026-05-11T14:30:03,248 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lug.yaml' 2026-05-11T14:30:03,249 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_orm.yaml' 2026-05-11T14:30:03,250 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sna.yaml' 2026-05-11T14:30:03,251 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sot.yaml' 2026-05-11T14:30:03,252 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_swa.yaml' 2026-05-11T14:30:03,254 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_twi.yaml' 2026-05-11T14:30:03,255 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_wol.yaml' 2026-05-11T14:30:03,256 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_xho.yaml' 2026-05-11T14:30:03,257 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_yor.yaml' 2026-05-11T14:30:03,258 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_zul.yaml' 2026-05-11T14:30:03,259 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/utils.py' 2026-05-11T14:30:03,261 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate' 2026-05-11T14:30:03,262 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_amh.yaml' 2026-05-11T14:30:03,263 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ewe.yaml' 2026-05-11T14:30:03,264 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_fra.yaml' 2026-05-11T14:30:03,265 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_hau.yaml' 2026-05-11T14:30:03,266 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ibo.yaml' 2026-05-11T14:30:03,267 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_kin.yaml' 2026-05-11T14:30:03,269 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lin.yaml' 2026-05-11T14:30:03,270 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lug.yaml' 2026-05-11T14:30:03,271 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_orm.yaml' 2026-05-11T14:30:03,272 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sna.yaml' 2026-05-11T14:30:03,273 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sot.yaml' 2026-05-11T14:30:03,274 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_swa.yaml' 2026-05-11T14:30:03,275 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_twi.yaml' 2026-05-11T14:30:03,276 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_wol.yaml' 2026-05-11T14:30:03,278 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_xho.yaml' 2026-05-11T14:30:03,279 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_yor.yaml' 2026-05-11T14:30:03,280 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_zul.yaml' 2026-05-11T14:30:03,281 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/utils.py' 2026-05-11T14:30:03,283 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate' 2026-05-11T14:30:03,285 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_amh.yaml' 2026-05-11T14:30:03,286 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ewe.yaml' 2026-05-11T14:30:03,287 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_fra.yaml' 2026-05-11T14:30:03,288 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_hau.yaml' 2026-05-11T14:30:03,290 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ibo.yaml' 2026-05-11T14:30:03,291 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_kin.yaml' 2026-05-11T14:30:03,292 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lin.yaml' 2026-05-11T14:30:03,293 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lug.yaml' 2026-05-11T14:30:03,294 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_orm.yaml' 2026-05-11T14:30:03,296 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sna.yaml' 2026-05-11T14:30:03,297 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sot.yaml' 2026-05-11T14:30:03,298 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_swa.yaml' 2026-05-11T14:30:03,299 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_twi.yaml' 2026-05-11T14:30:03,300 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_wol.yaml' 2026-05-11T14:30:03,301 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_xho.yaml' 2026-05-11T14:30:03,303 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_yor.yaml' 2026-05-11T14:30:03,304 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_zul.yaml' 2026-05-11T14:30:03,305 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/utils.py' 2026-05-11T14:30:03,307 adding 'lm_eval/tasks/afrixnli/README.md' 2026-05-11T14:30:03,308 adding 'lm_eval/tasks/afrixnli/gen_utils.py' 2026-05-11T14:30:03,309 adding 'lm_eval/tasks/afrixnli/utils.py' 2026-05-11T14:30:03,312 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_amh.yaml' 2026-05-11T14:30:03,313 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_eng.yaml' 2026-05-11T14:30:03,314 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ewe.yaml' 2026-05-11T14:30:03,315 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_fra.yaml' 2026-05-11T14:30:03,316 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_hau.yaml' 2026-05-11T14:30:03,317 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ibo.yaml' 2026-05-11T14:30:03,318 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_kin.yaml' 2026-05-11T14:30:03,319 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lin.yaml' 2026-05-11T14:30:03,320 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lug.yaml' 2026-05-11T14:30:03,321 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_orm.yaml' 2026-05-11T14:30:03,322 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sna.yaml' 2026-05-11T14:30:03,323 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sot.yaml' 2026-05-11T14:30:03,325 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_swa.yaml' 2026-05-11T14:30:03,326 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_twi.yaml' 2026-05-11T14:30:03,327 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_wol.yaml' 2026-05-11T14:30:03,328 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_xho.yaml' 2026-05-11T14:30:03,329 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yaml' 2026-05-11T14:30:03,330 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yor.yaml' 2026-05-11T14:30:03,331 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_zul.yaml' 2026-05-11T14:30:03,332 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/utils.py' 2026-05-11T14:30:03,334 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_amh.yaml' 2026-05-11T14:30:03,335 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_eng.yaml' 2026-05-11T14:30:03,337 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ewe.yaml' 2026-05-11T14:30:03,338 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_fra.yaml' 2026-05-11T14:30:03,339 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_hau.yaml' 2026-05-11T14:30:03,340 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ibo.yaml' 2026-05-11T14:30:03,341 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_kin.yaml' 2026-05-11T14:30:03,342 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lin.yaml' 2026-05-11T14:30:03,344 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lug.yaml' 2026-05-11T14:30:03,345 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_orm.yaml' 2026-05-11T14:30:03,346 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sna.yaml' 2026-05-11T14:30:03,347 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sot.yaml' 2026-05-11T14:30:03,349 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_swa.yaml' 2026-05-11T14:30:03,350 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_twi.yaml' 2026-05-11T14:30:03,351 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_wol.yaml' 2026-05-11T14:30:03,352 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_xho.yaml' 2026-05-11T14:30:03,353 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yaml' 2026-05-11T14:30:03,354 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yor.yaml' 2026-05-11T14:30:03,355 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_zul.yaml' 2026-05-11T14:30:03,356 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/utils.py' 2026-05-11T14:30:03,358 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_amh.yaml' 2026-05-11T14:30:03,359 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ewe.yaml' 2026-05-11T14:30:03,360 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_fra.yaml' 2026-05-11T14:30:03,361 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_hau.yaml' 2026-05-11T14:30:03,362 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ibo.yaml' 2026-05-11T14:30:03,363 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_kin.yaml' 2026-05-11T14:30:03,364 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lin.yaml' 2026-05-11T14:30:03,365 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lug.yaml' 2026-05-11T14:30:03,366 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_orm.yaml' 2026-05-11T14:30:03,368 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sna.yaml' 2026-05-11T14:30:03,369 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sot.yaml' 2026-05-11T14:30:03,370 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_swa.yaml' 2026-05-11T14:30:03,371 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_twi.yaml' 2026-05-11T14:30:03,372 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_wol.yaml' 2026-05-11T14:30:03,373 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_xho.yaml' 2026-05-11T14:30:03,375 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yaml' 2026-05-11T14:30:03,376 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yor.yaml' 2026-05-11T14:30:03,377 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_zul.yaml' 2026-05-11T14:30:03,378 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/utils.py' 2026-05-11T14:30:03,380 adding 'lm_eval/tasks/afrixnli/direct/afrixnli.yaml' 2026-05-11T14:30:03,382 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_amh.yaml' 2026-05-11T14:30:03,383 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_eng.yaml' 2026-05-11T14:30:03,384 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ewe.yaml' 2026-05-11T14:30:03,385 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_fra.yaml' 2026-05-11T14:30:03,387 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_hau.yaml' 2026-05-11T14:30:03,388 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ibo.yaml' 2026-05-11T14:30:03,389 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_kin.yaml' 2026-05-11T14:30:03,390 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lin.yaml' 2026-05-11T14:30:03,391 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lug.yaml' 2026-05-11T14:30:03,393 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_orm.yaml' 2026-05-11T14:30:03,394 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sna.yaml' 2026-05-11T14:30:03,395 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sot.yaml' 2026-05-11T14:30:03,396 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_swa.yaml' 2026-05-11T14:30:03,397 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_twi.yaml' 2026-05-11T14:30:03,398 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_wol.yaml' 2026-05-11T14:30:03,399 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_xho.yaml' 2026-05-11T14:30:03,400 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yaml' 2026-05-11T14:30:03,402 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yor.yaml' 2026-05-11T14:30:03,403 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_zul.yaml' 2026-05-11T14:30:03,404 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/utils.py' 2026-05-11T14:30:03,405 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_amh.yaml' 2026-05-11T14:30:03,407 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_eng.yaml' 2026-05-11T14:30:03,408 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ewe.yaml' 2026-05-11T14:30:03,409 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_fra.yaml' 2026-05-11T14:30:03,410 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_hau.yaml' 2026-05-11T14:30:03,411 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ibo.yaml' 2026-05-11T14:30:03,438 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_kin.yaml' 2026-05-11T14:30:03,439 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lin.yaml' 2026-05-11T14:30:03,440 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lug.yaml' 2026-05-11T14:30:03,441 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_orm.yaml' 2026-05-11T14:30:03,442 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sna.yaml' 2026-05-11T14:30:03,444 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sot.yaml' 2026-05-11T14:30:03,445 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_swa.yaml' 2026-05-11T14:30:03,446 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_twi.yaml' 2026-05-11T14:30:03,447 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_wol.yaml' 2026-05-11T14:30:03,448 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_xho.yaml' 2026-05-11T14:30:03,449 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yaml' 2026-05-11T14:30:03,451 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yor.yaml' 2026-05-11T14:30:03,452 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_zul.yaml' 2026-05-11T14:30:03,453 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/utils.py' 2026-05-11T14:30:03,455 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_amh.yaml' 2026-05-11T14:30:03,456 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_eng.yaml' 2026-05-11T14:30:03,457 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ewe.yaml' 2026-05-11T14:30:03,458 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_fra.yaml' 2026-05-11T14:30:03,460 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_hau.yaml' 2026-05-11T14:30:03,461 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ibo.yaml' 2026-05-11T14:30:03,462 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_kin.yaml' 2026-05-11T14:30:03,463 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lin.yaml' 2026-05-11T14:30:03,464 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lug.yaml' 2026-05-11T14:30:03,466 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_orm.yaml' 2026-05-11T14:30:03,467 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sna.yaml' 2026-05-11T14:30:03,468 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sot.yaml' 2026-05-11T14:30:03,469 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_swa.yaml' 2026-05-11T14:30:03,470 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_twi.yaml' 2026-05-11T14:30:03,472 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_wol.yaml' 2026-05-11T14:30:03,473 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_xho.yaml' 2026-05-11T14:30:03,474 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yaml' 2026-05-11T14:30:03,475 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yor.yaml' 2026-05-11T14:30:03,476 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_zul.yaml' 2026-05-11T14:30:03,477 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/utils.py' 2026-05-11T14:30:03,479 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_amh.yaml' 2026-05-11T14:30:03,480 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_eng.yaml' 2026-05-11T14:30:03,481 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ewe.yaml' 2026-05-11T14:30:03,482 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_fra.yaml' 2026-05-11T14:30:03,483 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_hau.yaml' 2026-05-11T14:30:03,485 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ibo.yaml' 2026-05-11T14:30:03,486 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_kin.yaml' 2026-05-11T14:30:03,487 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lin.yaml' 2026-05-11T14:30:03,488 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lug.yaml' 2026-05-11T14:30:03,489 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_orm.yaml' 2026-05-11T14:30:03,490 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sna.yaml' 2026-05-11T14:30:03,492 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sot.yaml' 2026-05-11T14:30:03,493 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_swa.yaml' 2026-05-11T14:30:03,494 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_twi.yaml' 2026-05-11T14:30:03,495 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_wol.yaml' 2026-05-11T14:30:03,496 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_xho.yaml' 2026-05-11T14:30:03,497 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yaml' 2026-05-11T14:30:03,498 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yor.yaml' 2026-05-11T14:30:03,500 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_zul.yaml' 2026-05-11T14:30:03,501 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/utils.py' 2026-05-11T14:30:03,503 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_amh.yaml' 2026-05-11T14:30:03,504 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_eng.yaml' 2026-05-11T14:30:03,505 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ewe.yaml' 2026-05-11T14:30:03,506 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_fra.yaml' 2026-05-11T14:30:03,508 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_hau.yaml' 2026-05-11T14:30:03,509 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ibo.yaml' 2026-05-11T14:30:03,510 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_kin.yaml' 2026-05-11T14:30:03,511 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lin.yaml' 2026-05-11T14:30:03,512 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lug.yaml' 2026-05-11T14:30:03,514 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_orm.yaml' 2026-05-11T14:30:03,515 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sna.yaml' 2026-05-11T14:30:03,516 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sot.yaml' 2026-05-11T14:30:03,517 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_swa.yaml' 2026-05-11T14:30:03,518 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_twi.yaml' 2026-05-11T14:30:03,520 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_wol.yaml' 2026-05-11T14:30:03,521 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_xho.yaml' 2026-05-11T14:30:03,522 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yaml' 2026-05-11T14:30:03,523 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yor.yaml' 2026-05-11T14:30:03,525 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_zul.yaml' 2026-05-11T14:30:03,526 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/utils.py' 2026-05-11T14:30:03,529 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_amh.yaml' 2026-05-11T14:30:03,530 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_eng.yaml' 2026-05-11T14:30:03,531 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ewe.yaml' 2026-05-11T14:30:03,532 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_fra.yaml' 2026-05-11T14:30:03,533 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_hau.yaml' 2026-05-11T14:30:03,534 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ibo.yaml' 2026-05-11T14:30:03,535 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_kin.yaml' 2026-05-11T14:30:03,536 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lin.yaml' 2026-05-11T14:30:03,537 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lug.yaml' 2026-05-11T14:30:03,538 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_orm.yaml' 2026-05-11T14:30:03,539 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sna.yaml' 2026-05-11T14:30:03,540 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sot.yaml' 2026-05-11T14:30:03,542 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_swa.yaml' 2026-05-11T14:30:03,543 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_twi.yaml' 2026-05-11T14:30:03,544 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_wol.yaml' 2026-05-11T14:30:03,545 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_xho.yaml' 2026-05-11T14:30:03,546 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yaml' 2026-05-11T14:30:03,547 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yor.yaml' 2026-05-11T14:30:03,548 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_zul.yaml' 2026-05-11T14:30:03,550 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/utils.py' 2026-05-11T14:30:03,551 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_amh.yaml' 2026-05-11T14:30:03,552 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ewe.yaml' 2026-05-11T14:30:03,554 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_fra.yaml' 2026-05-11T14:30:03,555 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_hau.yaml' 2026-05-11T14:30:03,556 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ibo.yaml' 2026-05-11T14:30:03,557 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_kin.yaml' 2026-05-11T14:30:03,558 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lin.yaml' 2026-05-11T14:30:03,559 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lug.yaml' 2026-05-11T14:30:03,560 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_orm.yaml' 2026-05-11T14:30:03,562 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sna.yaml' 2026-05-11T14:30:03,563 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sot.yaml' 2026-05-11T14:30:03,564 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_swa.yaml' 2026-05-11T14:30:03,565 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_twi.yaml' 2026-05-11T14:30:03,566 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_wol.yaml' 2026-05-11T14:30:03,567 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_xho.yaml' 2026-05-11T14:30:03,568 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yaml' 2026-05-11T14:30:03,570 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yor.yaml' 2026-05-11T14:30:03,571 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_zul.yaml' 2026-05-11T14:30:03,572 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/utils.py' 2026-05-11T14:30:03,574 adding 'lm_eval/tasks/afrixnli/translate/afrixnli_tt.yaml' 2026-05-11T14:30:03,576 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_amh.yaml' 2026-05-11T14:30:03,577 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ewe.yaml' 2026-05-11T14:30:03,578 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_fra.yaml' 2026-05-11T14:30:03,579 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_hau.yaml' 2026-05-11T14:30:03,580 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ibo.yaml' 2026-05-11T14:30:03,581 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_kin.yaml' 2026-05-11T14:30:03,582 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lin.yaml' 2026-05-11T14:30:03,583 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lug.yaml' 2026-05-11T14:30:03,584 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_orm.yaml' 2026-05-11T14:30:03,586 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sna.yaml' 2026-05-11T14:30:03,587 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sot.yaml' 2026-05-11T14:30:03,588 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_swa.yaml' 2026-05-11T14:30:03,589 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_twi.yaml' 2026-05-11T14:30:03,591 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_wol.yaml' 2026-05-11T14:30:03,592 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_xho.yaml' 2026-05-11T14:30:03,593 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yaml' 2026-05-11T14:30:03,594 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yor.yaml' 2026-05-11T14:30:03,595 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_zul.yaml' 2026-05-11T14:30:03,597 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/utils.py' 2026-05-11T14:30:03,599 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_amh.yaml' 2026-05-11T14:30:03,600 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ewe.yaml' 2026-05-11T14:30:03,601 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_fra.yaml' 2026-05-11T14:30:03,602 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_hau.yaml' 2026-05-11T14:30:03,603 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ibo.yaml' 2026-05-11T14:30:03,604 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_kin.yaml' 2026-05-11T14:30:03,605 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lin.yaml' 2026-05-11T14:30:03,607 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lug.yaml' 2026-05-11T14:30:03,608 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_orm.yaml' 2026-05-11T14:30:03,609 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sna.yaml' 2026-05-11T14:30:03,610 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sot.yaml' 2026-05-11T14:30:03,611 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_swa.yaml' 2026-05-11T14:30:03,613 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_twi.yaml' 2026-05-11T14:30:03,614 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_wol.yaml' 2026-05-11T14:30:03,615 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_xho.yaml' 2026-05-11T14:30:03,616 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yaml' 2026-05-11T14:30:03,617 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yor.yaml' 2026-05-11T14:30:03,619 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_zul.yaml' 2026-05-11T14:30:03,620 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/utils.py' 2026-05-11T14:30:03,622 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_amh.yaml' 2026-05-11T14:30:03,623 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ewe.yaml' 2026-05-11T14:30:03,624 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_fra.yaml' 2026-05-11T14:30:03,625 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_hau.yaml' 2026-05-11T14:30:03,626 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ibo.yaml' 2026-05-11T14:30:03,627 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_kin.yaml' 2026-05-11T14:30:03,628 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lin.yaml' 2026-05-11T14:30:03,629 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lug.yaml' 2026-05-11T14:30:03,630 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_orm.yaml' 2026-05-11T14:30:03,631 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sna.yaml' 2026-05-11T14:30:03,633 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sot.yaml' 2026-05-11T14:30:03,634 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_swa.yaml' 2026-05-11T14:30:03,635 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_twi.yaml' 2026-05-11T14:30:03,636 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_wol.yaml' 2026-05-11T14:30:03,637 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_xho.yaml' 2026-05-11T14:30:03,639 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yaml' 2026-05-11T14:30:03,640 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yor.yaml' 2026-05-11T14:30:03,641 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_zul.yaml' 2026-05-11T14:30:03,642 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/utils.py' 2026-05-11T14:30:03,644 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_amh.yaml' 2026-05-11T14:30:03,645 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ewe.yaml' 2026-05-11T14:30:03,647 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_fra.yaml' 2026-05-11T14:30:03,648 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_hau.yaml' 2026-05-11T14:30:03,649 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ibo.yaml' 2026-05-11T14:30:03,650 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_kin.yaml' 2026-05-11T14:30:03,652 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lin.yaml' 2026-05-11T14:30:03,653 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lug.yaml' 2026-05-11T14:30:03,654 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_orm.yaml' 2026-05-11T14:30:03,655 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sna.yaml' 2026-05-11T14:30:03,656 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sot.yaml' 2026-05-11T14:30:03,658 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_swa.yaml' 2026-05-11T14:30:03,659 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_twi.yaml' 2026-05-11T14:30:03,660 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_wol.yaml' 2026-05-11T14:30:03,661 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_xho.yaml' 2026-05-11T14:30:03,663 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yaml' 2026-05-11T14:30:03,664 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yor.yaml' 2026-05-11T14:30:03,665 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_zul.yaml' 2026-05-11T14:30:03,667 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/utils.py' 2026-05-11T14:30:03,668 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_amh.yaml' 2026-05-11T14:30:03,670 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ewe.yaml' 2026-05-11T14:30:03,671 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_fra.yaml' 2026-05-11T14:30:03,672 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_hau.yaml' 2026-05-11T14:30:03,673 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ibo.yaml' 2026-05-11T14:30:03,674 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_kin.yaml' 2026-05-11T14:30:03,675 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lin.yaml' 2026-05-11T14:30:03,676 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lug.yaml' 2026-05-11T14:30:03,677 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_orm.yaml' 2026-05-11T14:30:03,678 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sna.yaml' 2026-05-11T14:30:03,680 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sot.yaml' 2026-05-11T14:30:03,681 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_swa.yaml' 2026-05-11T14:30:03,682 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_twi.yaml' 2026-05-11T14:30:03,683 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_wol.yaml' 2026-05-11T14:30:03,684 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_xho.yaml' 2026-05-11T14:30:03,685 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yaml' 2026-05-11T14:30:03,686 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yor.yaml' 2026-05-11T14:30:03,687 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_zul.yaml' 2026-05-11T14:30:03,688 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/utils.py' 2026-05-11T14:30:03,691 adding 'lm_eval/tasks/afrobench/README.md' 2026-05-11T14:30:03,692 adding 'lm_eval/tasks/afrobench/afrobench-lite.yaml' 2026-05-11T14:30:03,693 adding 'lm_eval/tasks/afrobench/afrobench.yaml' 2026-05-11T14:30:03,695 adding 'lm_eval/tasks/afrobench/adr/README.md' 2026-05-11T14:30:03,696 adding 'lm_eval/tasks/afrobench/adr/afridiacritics.yaml' 2026-05-11T14:30:03,697 adding 'lm_eval/tasks/afrobench/adr/gen_utils.py' 2026-05-11T14:30:03,699 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_bbj.yaml' 2026-05-11T14:30:03,700 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_fon.yaml' 2026-05-11T14:30:03,702 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_ibo.yaml' 2026-05-11T14:30:03,703 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_wol.yaml' 2026-05-11T14:30:03,704 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yaml' 2026-05-11T14:30:03,705 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yor.yaml' 2026-05-11T14:30:03,707 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_bbj.yaml' 2026-05-11T14:30:03,708 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_fon.yaml' 2026-05-11T14:30:03,710 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_ibo.yaml' 2026-05-11T14:30:03,711 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_wol.yaml' 2026-05-11T14:30:03,712 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yaml' 2026-05-11T14:30:03,713 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yor.yaml' 2026-05-11T14:30:03,715 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_bbj.yaml' 2026-05-11T14:30:03,716 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_fon.yaml' 2026-05-11T14:30:03,718 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_ibo.yaml' 2026-05-11T14:30:03,719 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_wol.yaml' 2026-05-11T14:30:03,720 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yaml' 2026-05-11T14:30:03,721 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yor.yaml' 2026-05-11T14:30:03,723 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_bbj.yaml' 2026-05-11T14:30:03,724 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_fon.yaml' 2026-05-11T14:30:03,725 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_ibo.yaml' 2026-05-11T14:30:03,726 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_wol.yaml' 2026-05-11T14:30:03,727 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yaml' 2026-05-11T14:30:03,729 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yor.yaml' 2026-05-11T14:30:03,730 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_bbj.yaml' 2026-05-11T14:30:03,731 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_fon.yaml' 2026-05-11T14:30:03,733 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_ibo.yaml' 2026-05-11T14:30:03,734 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_wol.yaml' 2026-05-11T14:30:03,735 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yaml' 2026-05-11T14:30:03,736 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yor.yaml' 2026-05-11T14:30:03,738 adding 'lm_eval/tasks/afrobench/afriqa/README.md' 2026-05-11T14:30:03,739 adding 'lm_eval/tasks/afrobench/afriqa/afriqa.yaml' 2026-05-11T14:30:03,740 adding 'lm_eval/tasks/afrobench/afriqa/utils.py' 2026-05-11T14:30:03,742 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa' 2026-05-11T14:30:03,744 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_bem.yaml' 2026-05-11T14:30:03,745 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_fon.yaml' 2026-05-11T14:30:03,746 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_hau.yaml' 2026-05-11T14:30:03,747 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_ibo.yaml' 2026-05-11T14:30:03,749 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_kin.yaml' 2026-05-11T14:30:03,750 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_swa.yaml' 2026-05-11T14:30:03,751 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_twi.yaml' 2026-05-11T14:30:03,752 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_yor.yaml' 2026-05-11T14:30:03,753 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_zul.yaml' 2026-05-11T14:30:03,755 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/utils.py' 2026-05-11T14:30:03,757 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa' 2026-05-11T14:30:03,758 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_bem.yaml' 2026-05-11T14:30:03,759 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_fon.yaml' 2026-05-11T14:30:03,760 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_hau.yaml' 2026-05-11T14:30:03,761 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_ibo.yaml' 2026-05-11T14:30:03,762 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_kin.yaml' 2026-05-11T14:30:03,764 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_swa.yaml' 2026-05-11T14:30:03,765 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_twi.yaml' 2026-05-11T14:30:03,766 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_yor.yaml' 2026-05-11T14:30:03,767 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_zul.yaml' 2026-05-11T14:30:03,769 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/utils.py' 2026-05-11T14:30:03,770 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa' 2026-05-11T14:30:03,772 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_bem.yaml' 2026-05-11T14:30:03,773 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_fon.yaml' 2026-05-11T14:30:03,774 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_hau.yaml' 2026-05-11T14:30:03,775 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_ibo.yaml' 2026-05-11T14:30:03,776 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_kin.yaml' 2026-05-11T14:30:03,778 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_swa.yaml' 2026-05-11T14:30:03,779 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_twi.yaml' 2026-05-11T14:30:03,780 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_yor.yaml' 2026-05-11T14:30:03,781 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_zul.yaml' 2026-05-11T14:30:03,782 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/utils.py' 2026-05-11T14:30:03,784 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa' 2026-05-11T14:30:03,785 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_bem.yaml' 2026-05-11T14:30:03,786 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_fon.yaml' 2026-05-11T14:30:03,787 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_hau.yaml' 2026-05-11T14:30:03,788 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_ibo.yaml' 2026-05-11T14:30:03,789 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_kin.yaml' 2026-05-11T14:30:03,791 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_swa.yaml' 2026-05-11T14:30:03,792 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_twi.yaml' 2026-05-11T14:30:03,793 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_yor.yaml' 2026-05-11T14:30:03,794 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_zul.yaml' 2026-05-11T14:30:03,795 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/utils.py' 2026-05-11T14:30:03,797 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa' 2026-05-11T14:30:03,798 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_bem.yaml' 2026-05-11T14:30:03,800 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_fon.yaml' 2026-05-11T14:30:03,801 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_hau.yaml' 2026-05-11T14:30:03,802 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_ibo.yaml' 2026-05-11T14:30:03,803 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_kin.yaml' 2026-05-11T14:30:03,804 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_swa.yaml' 2026-05-11T14:30:03,805 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_twi.yaml' 2026-05-11T14:30:03,807 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_yor.yaml' 2026-05-11T14:30:03,808 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_zul.yaml' 2026-05-11T14:30:03,809 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/utils.py' 2026-05-11T14:30:03,811 adding 'lm_eval/tasks/afrobench/afrisenti/README.md' 2026-05-11T14:30:03,812 adding 'lm_eval/tasks/afrobench/afrisenti/afrisenti.yaml' 2026-05-11T14:30:03,814 adding 'lm_eval/tasks/afrobench/afrisenti/fewshot.sh' 2026-05-11T14:30:03,815 adding 'lm_eval/tasks/afrobench/afrisenti/utils.py' 2026-05-11T14:30:03,817 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti' 2026-05-11T14:30:03,818 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_amh.yaml' 2026-05-11T14:30:03,820 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_arq.yaml' 2026-05-11T14:30:03,821 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ary.yaml' 2026-05-11T14:30:03,822 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_hau.yaml' 2026-05-11T14:30:03,823 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ibo.yaml' 2026-05-11T14:30:03,824 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_kin.yaml' 2026-05-11T14:30:03,825 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_orm.yaml' 2026-05-11T14:30:03,826 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_pcm.yaml' 2026-05-11T14:30:03,828 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_por.yaml' 2026-05-11T14:30:03,829 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_swa.yaml' 2026-05-11T14:30:03,830 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tir.yaml' 2026-05-11T14:30:03,831 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tso.yaml' 2026-05-11T14:30:03,832 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_twi.yaml' 2026-05-11T14:30:03,833 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_yor.yaml' 2026-05-11T14:30:03,835 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/run.sh' 2026-05-11T14:30:03,836 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/utils.py' 2026-05-11T14:30:03,837 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/xx.py' 2026-05-11T14:30:03,839 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti' 2026-05-11T14:30:03,840 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_amh.yaml' 2026-05-11T14:30:03,841 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_arq.yaml' 2026-05-11T14:30:03,842 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ary.yaml' 2026-05-11T14:30:03,844 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_hau.yaml' 2026-05-11T14:30:03,845 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ibo.yaml' 2026-05-11T14:30:03,846 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_kin.yaml' 2026-05-11T14:30:03,847 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_orm.yaml' 2026-05-11T14:30:03,848 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_pcm.yaml' 2026-05-11T14:30:03,849 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_por.yaml' 2026-05-11T14:30:03,850 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_swa.yaml' 2026-05-11T14:30:03,851 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tir.yaml' 2026-05-11T14:30:03,853 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tso.yaml' 2026-05-11T14:30:03,854 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_twi.yaml' 2026-05-11T14:30:03,855 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_yor.yaml' 2026-05-11T14:30:03,857 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/run.sh' 2026-05-11T14:30:03,858 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/utils.py' 2026-05-11T14:30:03,859 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/xx.py' 2026-05-11T14:30:03,861 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti' 2026-05-11T14:30:03,862 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_amh.yaml' 2026-05-11T14:30:03,863 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_arq.yaml' 2026-05-11T14:30:03,864 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ary.yaml' 2026-05-11T14:30:03,865 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_hau.yaml' 2026-05-11T14:30:03,867 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ibo.yaml' 2026-05-11T14:30:03,868 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_kin.yaml' 2026-05-11T14:30:03,869 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_orm.yaml' 2026-05-11T14:30:03,870 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_pcm.yaml' 2026-05-11T14:30:03,871 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_por.yaml' 2026-05-11T14:30:03,873 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_swa.yaml' 2026-05-11T14:30:03,874 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tir.yaml' 2026-05-11T14:30:03,875 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tso.yaml' 2026-05-11T14:30:03,876 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_twi.yaml' 2026-05-11T14:30:03,877 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_yor.yaml' 2026-05-11T14:30:03,878 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/utils.py' 2026-05-11T14:30:03,880 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/xx.py' 2026-05-11T14:30:03,881 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti' 2026-05-11T14:30:03,883 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_amh.yaml' 2026-05-11T14:30:03,884 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_arq.yaml' 2026-05-11T14:30:03,885 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ary.yaml' 2026-05-11T14:30:03,886 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_hau.yaml' 2026-05-11T14:30:03,887 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ibo.yaml' 2026-05-11T14:30:03,889 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_kin.yaml' 2026-05-11T14:30:03,890 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_orm.yaml' 2026-05-11T14:30:03,891 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_pcm.yaml' 2026-05-11T14:30:03,892 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_por.yaml' 2026-05-11T14:30:03,893 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_swa.yaml' 2026-05-11T14:30:03,894 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tir.yaml' 2026-05-11T14:30:03,895 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tso.yaml' 2026-05-11T14:30:03,897 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_twi.yaml' 2026-05-11T14:30:03,898 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_yor.yaml' 2026-05-11T14:30:03,899 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/utils.py' 2026-05-11T14:30:03,900 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/xx.py' 2026-05-11T14:30:03,902 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti' 2026-05-11T14:30:03,903 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_amh.yaml' 2026-05-11T14:30:03,904 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_arq.yaml' 2026-05-11T14:30:03,906 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ary.yaml' 2026-05-11T14:30:03,907 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_hau.yaml' 2026-05-11T14:30:03,908 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ibo.yaml' 2026-05-11T14:30:03,909 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_kin.yaml' 2026-05-11T14:30:03,910 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_orm.yaml' 2026-05-11T14:30:03,912 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_pcm.yaml' 2026-05-11T14:30:03,913 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_por.yaml' 2026-05-11T14:30:03,914 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_swa.yaml' 2026-05-11T14:30:03,915 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tir.yaml' 2026-05-11T14:30:03,917 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tso.yaml' 2026-05-11T14:30:03,918 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_twi.yaml' 2026-05-11T14:30:03,919 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_yor.yaml' 2026-05-11T14:30:03,920 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/utils.py' 2026-05-11T14:30:03,921 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/xx.py' 2026-05-11T14:30:03,923 adding 'lm_eval/tasks/afrobench/belebele/README.md' 2026-05-11T14:30:03,924 adding 'lm_eval/tasks/afrobench/belebele/belebele.yaml' 2026-05-11T14:30:03,926 adding 'lm_eval/tasks/afrobench/belebele/utils.py' 2026-05-11T14:30:03,928 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele' 2026-05-11T14:30:03,929 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_afr.yaml' 2026-05-11T14:30:03,930 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_amh.yaml' 2026-05-11T14:30:03,931 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ary.yaml' 2026-05-11T14:30:03,933 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_arz.yaml' 2026-05-11T14:30:03,934 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_bam.yaml' 2026-05-11T14:30:03,935 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_eng.yaml' 2026-05-11T14:30:03,936 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fra.yaml' 2026-05-11T14:30:03,938 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fuv.yaml' 2026-05-11T14:30:03,939 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_gaz.yaml' 2026-05-11T14:30:03,940 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_hau.yaml' 2026-05-11T14:30:03,941 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ibo.yaml' 2026-05-11T14:30:03,942 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kea.yaml' 2026-05-11T14:30:03,944 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kin.yaml' 2026-05-11T14:30:03,945 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lin.yaml' 2026-05-11T14:30:03,946 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lug.yaml' 2026-05-11T14:30:03,947 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_luo.yaml' 2026-05-11T14:30:03,948 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_nya.yaml' 2026-05-11T14:30:03,949 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_plt.yaml' 2026-05-11T14:30:03,950 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_por.yaml' 2026-05-11T14:30:03,951 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sna.yaml' 2026-05-11T14:30:03,953 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_som.yaml' 2026-05-11T14:30:03,954 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sot.yaml' 2026-05-11T14:30:03,955 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ssw.yaml' 2026-05-11T14:30:03,956 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_swa.yaml' 2026-05-11T14:30:03,957 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tir.yaml' 2026-05-11T14:30:03,958 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tsn.yaml' 2026-05-11T14:30:03,959 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tso.yaml' 2026-05-11T14:30:03,961 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_wol.yaml' 2026-05-11T14:30:03,962 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_xho.yaml' 2026-05-11T14:30:03,963 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_yor.yaml' 2026-05-11T14:30:03,964 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_zul.yaml' 2026-05-11T14:30:03,966 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele' 2026-05-11T14:30:03,968 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_afr.yaml' 2026-05-11T14:30:03,969 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_amh.yaml' 2026-05-11T14:30:03,970 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ary.yaml' 2026-05-11T14:30:03,971 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_arz.yaml' 2026-05-11T14:30:03,972 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_bam.yaml' 2026-05-11T14:30:03,974 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_eng.yaml' 2026-05-11T14:30:03,975 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fra.yaml' 2026-05-11T14:30:03,976 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fuv.yaml' 2026-05-11T14:30:03,977 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_gaz.yaml' 2026-05-11T14:30:03,978 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_hau.yaml' 2026-05-11T14:30:03,979 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ibo.yaml' 2026-05-11T14:30:03,981 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kea.yaml' 2026-05-11T14:30:03,982 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kin.yaml' 2026-05-11T14:30:03,983 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lin.yaml' 2026-05-11T14:30:03,984 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lug.yaml' 2026-05-11T14:30:03,985 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_luo.yaml' 2026-05-11T14:30:03,986 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_nya.yaml' 2026-05-11T14:30:03,988 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_plt.yaml' 2026-05-11T14:30:03,989 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_por.yaml' 2026-05-11T14:30:03,990 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sna.yaml' 2026-05-11T14:30:03,995 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_som.yaml' 2026-05-11T14:30:03,996 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sot.yaml' 2026-05-11T14:30:03,997 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ssw.yaml' 2026-05-11T14:30:03,999 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_swa.yaml' 2026-05-11T14:30:04,000 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tir.yaml' 2026-05-11T14:30:04,001 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tsn.yaml' 2026-05-11T14:30:04,003 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tso.yaml' 2026-05-11T14:30:04,004 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_wol.yaml' 2026-05-11T14:30:04,005 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_xho.yaml' 2026-05-11T14:30:04,007 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_yor.yaml' 2026-05-11T14:30:04,008 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_zul.yaml' 2026-05-11T14:30:04,010 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele' 2026-05-11T14:30:04,012 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_afr.yaml' 2026-05-11T14:30:04,013 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_amh.yaml' 2026-05-11T14:30:04,014 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ary.yaml' 2026-05-11T14:30:04,015 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_arz.yaml' 2026-05-11T14:30:04,016 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_bam.yaml' 2026-05-11T14:30:04,017 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_eng.yaml' 2026-05-11T14:30:04,018 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fra.yaml' 2026-05-11T14:30:04,020 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fuv.yaml' 2026-05-11T14:30:04,021 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_gaz.yaml' 2026-05-11T14:30:04,022 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_hau.yaml' 2026-05-11T14:30:04,023 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ibo.yaml' 2026-05-11T14:30:04,024 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kea.yaml' 2026-05-11T14:30:04,025 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kin.yaml' 2026-05-11T14:30:04,027 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lin.yaml' 2026-05-11T14:30:04,028 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lug.yaml' 2026-05-11T14:30:04,029 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_luo.yaml' 2026-05-11T14:30:04,030 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_nya.yaml' 2026-05-11T14:30:04,031 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_plt.yaml' 2026-05-11T14:30:04,033 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_por.yaml' 2026-05-11T14:30:04,034 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sna.yaml' 2026-05-11T14:30:04,035 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_som.yaml' 2026-05-11T14:30:04,037 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sot.yaml' 2026-05-11T14:30:04,038 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ssw.yaml' 2026-05-11T14:30:04,039 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_swa.yaml' 2026-05-11T14:30:04,041 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tir.yaml' 2026-05-11T14:30:04,042 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tsn.yaml' 2026-05-11T14:30:04,043 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tso.yaml' 2026-05-11T14:30:04,044 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_wol.yaml' 2026-05-11T14:30:04,045 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_xho.yaml' 2026-05-11T14:30:04,046 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_yor.yaml' 2026-05-11T14:30:04,048 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_zul.yaml' 2026-05-11T14:30:04,050 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele' 2026-05-11T14:30:04,051 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_afr.yaml' 2026-05-11T14:30:04,052 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_amh.yaml' 2026-05-11T14:30:04,053 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ary.yaml' 2026-05-11T14:30:04,055 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_arz.yaml' 2026-05-11T14:30:04,056 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_bam.yaml' 2026-05-11T14:30:04,057 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_eng.yaml' 2026-05-11T14:30:04,058 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fra.yaml' 2026-05-11T14:30:04,061 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fuv.yaml' 2026-05-11T14:30:04,062 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_gaz.yaml' 2026-05-11T14:30:04,063 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_hau.yaml' 2026-05-11T14:30:04,064 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ibo.yaml' 2026-05-11T14:30:04,065 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kea.yaml' 2026-05-11T14:30:04,066 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kin.yaml' 2026-05-11T14:30:04,068 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lin.yaml' 2026-05-11T14:30:04,069 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lug.yaml' 2026-05-11T14:30:04,070 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_luo.yaml' 2026-05-11T14:30:04,071 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_nya.yaml' 2026-05-11T14:30:04,072 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_plt.yaml' 2026-05-11T14:30:04,074 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_por.yaml' 2026-05-11T14:30:04,075 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sna.yaml' 2026-05-11T14:30:04,076 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_som.yaml' 2026-05-11T14:30:04,077 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sot.yaml' 2026-05-11T14:30:04,078 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ssw.yaml' 2026-05-11T14:30:04,079 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_swa.yaml' 2026-05-11T14:30:04,081 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tir.yaml' 2026-05-11T14:30:04,082 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tsn.yaml' 2026-05-11T14:30:04,083 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tso.yaml' 2026-05-11T14:30:04,084 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_wol.yaml' 2026-05-11T14:30:04,085 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_xho.yaml' 2026-05-11T14:30:04,087 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_yor.yaml' 2026-05-11T14:30:04,088 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_zul.yaml' 2026-05-11T14:30:04,090 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele' 2026-05-11T14:30:04,091 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_afr.yaml' 2026-05-11T14:30:04,093 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_amh.yaml' 2026-05-11T14:30:04,094 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ary.yaml' 2026-05-11T14:30:04,096 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_arz.yaml' 2026-05-11T14:30:04,097 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_bam.yaml' 2026-05-11T14:30:04,098 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_eng.yaml' 2026-05-11T14:30:04,099 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fra.yaml' 2026-05-11T14:30:04,101 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fuv.yaml' 2026-05-11T14:30:04,102 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_gaz.yaml' 2026-05-11T14:30:04,103 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_hau.yaml' 2026-05-11T14:30:04,104 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ibo.yaml' 2026-05-11T14:30:04,106 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kea.yaml' 2026-05-11T14:30:04,107 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kin.yaml' 2026-05-11T14:30:04,108 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lin.yaml' 2026-05-11T14:30:04,109 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lug.yaml' 2026-05-11T14:30:04,110 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_luo.yaml' 2026-05-11T14:30:04,111 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_nya.yaml' 2026-05-11T14:30:04,113 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_plt.yaml' 2026-05-11T14:30:04,114 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_por.yaml' 2026-05-11T14:30:04,115 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sna.yaml' 2026-05-11T14:30:04,116 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_som.yaml' 2026-05-11T14:30:04,117 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sot.yaml' 2026-05-11T14:30:04,118 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ssw.yaml' 2026-05-11T14:30:04,119 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_swa.yaml' 2026-05-11T14:30:04,121 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tir.yaml' 2026-05-11T14:30:04,122 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tsn.yaml' 2026-05-11T14:30:04,123 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tso.yaml' 2026-05-11T14:30:04,124 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_wol.yaml' 2026-05-11T14:30:04,125 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_xho.yaml' 2026-05-11T14:30:04,126 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_yor.yaml' 2026-05-11T14:30:04,128 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_zul.yaml' 2026-05-11T14:30:04,130 adding 'lm_eval/tasks/afrobench/flores/README.md' 2026-05-11T14:30:04,131 adding 'lm_eval/tasks/afrobench/flores/flores.yaml' 2026-05-11T14:30:04,132 adding 'lm_eval/tasks/afrobench/flores/gen_utils.py' 2026-05-11T14:30:04,134 adding 'lm_eval/tasks/afrobench/flores/prompt_1/flores' 2026-05-11T14:30:04,137 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores' 2026-05-11T14:30:04,138 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,139 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,140 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_acq_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,141 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aeb_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,142 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_afr_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,144 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aka_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,145 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_amh_Ethi-eng_Latn.yaml' 2026-05-11T14:30:04,146 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ary_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,147 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_arz_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,148 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bam_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,149 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ban_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,150 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bem_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,152 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_cjk_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,154 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dik_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,155 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dyu_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,156 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ewe_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,157 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fon_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,158 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fra_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,160 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fuv_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,161 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_gaz_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,162 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_hau_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,163 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ibo_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,164 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kab_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,166 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kam_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,167 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kbp_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,168 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kea_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,169 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kik_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,170 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kin_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,171 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kmb_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,173 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,174 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,175 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kon_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,176 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lin_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,177 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lua_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,179 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lug_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,180 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_luo_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,181 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_mos_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,182 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nso_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,183 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nus_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,184 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nya_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,186 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_plt_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,188 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_run_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,189 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sag_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,190 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sna_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,192 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_som_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,193 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sot_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,194 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ssw_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,195 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sun_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,196 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_swh_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,197 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,199 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Tfng-eng_Latn.yaml' 2026-05-11T14:30:04,200 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tir_Ethi-eng_Latn.yaml' 2026-05-11T14:30:04,201 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tsn_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,202 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tso_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,203 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tum_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,204 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_twi_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,205 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tzm_Tfng-eng_Latn.yaml' 2026-05-11T14:30:04,206 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_umb_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,207 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_wol_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,208 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_xho_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,209 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_yor_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,211 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_zul_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,213 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores' 2026-05-11T14:30:04,214 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Arab.yaml' 2026-05-11T14:30:04,216 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Latn.yaml' 2026-05-11T14:30:04,217 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-acq_Arab.yaml' 2026-05-11T14:30:04,218 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aeb_Arab.yaml' 2026-05-11T14:30:04,219 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-afr_Latn.yaml' 2026-05-11T14:30:04,220 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aka_Latn.yaml' 2026-05-11T14:30:04,221 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-amh_Ethi.yaml' 2026-05-11T14:30:04,222 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ary_Arab.yaml' 2026-05-11T14:30:04,223 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-arz_Arab.yaml' 2026-05-11T14:30:04,224 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bam_Latn.yaml' 2026-05-11T14:30:04,226 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ban_Latn.yaml' 2026-05-11T14:30:04,227 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bem_Latn.yaml' 2026-05-11T14:30:04,228 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-cjk_Latn.yaml' 2026-05-11T14:30:04,229 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dik_Latn.yaml' 2026-05-11T14:30:04,230 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dyu_Latn.yaml' 2026-05-11T14:30:04,231 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ewe_Latn.yaml' 2026-05-11T14:30:04,232 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fon_Latn.yaml' 2026-05-11T14:30:04,234 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fra_Latn.yaml' 2026-05-11T14:30:04,235 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fuv_Latn.yaml' 2026-05-11T14:30:04,236 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-gaz_Latn.yaml' 2026-05-11T14:30:04,237 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-hau_Latn.yaml' 2026-05-11T14:30:04,238 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ibo_Latn.yaml' 2026-05-11T14:30:04,239 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kab_Latn.yaml' 2026-05-11T14:30:04,241 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kam_Latn.yaml' 2026-05-11T14:30:04,242 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kbp_Latn.yaml' 2026-05-11T14:30:04,243 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kea_Latn.yaml' 2026-05-11T14:30:04,244 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kik_Latn.yaml' 2026-05-11T14:30:04,245 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kin_Latn.yaml' 2026-05-11T14:30:04,246 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kmb_Latn.yaml' 2026-05-11T14:30:04,247 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Arab.yaml' 2026-05-11T14:30:04,248 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Latn.yaml' 2026-05-11T14:30:04,249 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kon_Latn.yaml' 2026-05-11T14:30:04,251 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lin_Latn.yaml' 2026-05-11T14:30:04,252 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lua_Latn.yaml' 2026-05-11T14:30:04,253 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lug_Latn.yaml' 2026-05-11T14:30:04,254 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-luo_Latn.yaml' 2026-05-11T14:30:04,255 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-mos_Latn.yaml' 2026-05-11T14:30:04,256 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nso_Latn.yaml' 2026-05-11T14:30:04,257 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nus_Latn.yaml' 2026-05-11T14:30:04,259 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nya_Latn.yaml' 2026-05-11T14:30:04,260 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-plt_Latn.yaml' 2026-05-11T14:30:04,261 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-run_Latn.yaml' 2026-05-11T14:30:04,262 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sag_Latn.yaml' 2026-05-11T14:30:04,263 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sna_Latn.yaml' 2026-05-11T14:30:04,264 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-som_Latn.yaml' 2026-05-11T14:30:04,265 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sot_Latn.yaml' 2026-05-11T14:30:04,267 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ssw_Latn.yaml' 2026-05-11T14:30:04,268 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sun_Latn.yaml' 2026-05-11T14:30:04,269 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-swh_Latn.yaml' 2026-05-11T14:30:04,270 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Latn.yaml' 2026-05-11T14:30:04,271 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Tfng.yaml' 2026-05-11T14:30:04,272 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tir_Ethi.yaml' 2026-05-11T14:30:04,273 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tsn_Latn.yaml' 2026-05-11T14:30:04,275 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tso_Latn.yaml' 2026-05-11T14:30:04,276 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tum_Latn.yaml' 2026-05-11T14:30:04,277 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-twi_Latn.yaml' 2026-05-11T14:30:04,278 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tzm_Tfng.yaml' 2026-05-11T14:30:04,279 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-umb_Latn.yaml' 2026-05-11T14:30:04,280 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-wol_Latn.yaml' 2026-05-11T14:30:04,281 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-xho_Latn.yaml' 2026-05-11T14:30:04,283 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-yor_Latn.yaml' 2026-05-11T14:30:04,284 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-zul_Latn.yaml' 2026-05-11T14:30:04,285 adding 'lm_eval/tasks/afrobench/flores/prompt_2/flores' 2026-05-11T14:30:04,288 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores' 2026-05-11T14:30:04,289 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,290 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,291 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_acq_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,293 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aeb_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,294 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_afr_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,295 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aka_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,296 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_amh_Ethi-eng_Latn.yaml' 2026-05-11T14:30:04,297 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ary_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,298 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_arz_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,299 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bam_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,301 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ban_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,302 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bem_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,303 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_cjk_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,304 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dik_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,305 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dyu_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,306 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ewe_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,307 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fon_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,309 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fra_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,310 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fuv_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,311 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_gaz_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,312 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_hau_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,313 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ibo_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,314 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kab_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,315 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kam_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,317 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kbp_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,318 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kea_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,319 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kik_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,320 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kin_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,321 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kmb_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,323 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,324 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,325 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kon_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,326 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lin_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,327 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lua_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,328 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lug_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,329 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_luo_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,330 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_mos_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,331 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nso_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,332 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nus_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,334 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nya_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,335 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_plt_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,336 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_run_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,337 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sag_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,338 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sna_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,339 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_som_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,340 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sot_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,341 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ssw_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,343 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sun_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,344 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_swh_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,345 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,346 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Tfng-eng_Latn.yaml' 2026-05-11T14:30:04,347 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tir_Ethi-eng_Latn.yaml' 2026-05-11T14:30:04,348 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tsn_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,350 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tso_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,351 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tum_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,352 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_twi_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,353 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tzm_Tfng-eng_Latn.yaml' 2026-05-11T14:30:04,354 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_umb_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,356 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_wol_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,357 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_xho_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,358 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_yor_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,359 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_zul_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,362 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores' 2026-05-11T14:30:04,363 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Arab.yaml' 2026-05-11T14:30:04,364 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Latn.yaml' 2026-05-11T14:30:04,365 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-acq_Arab.yaml' 2026-05-11T14:30:04,366 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aeb_Arab.yaml' 2026-05-11T14:30:04,367 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-afr_Latn.yaml' 2026-05-11T14:30:04,368 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aka_Latn.yaml' 2026-05-11T14:30:04,369 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-amh_Ethi.yaml' 2026-05-11T14:30:04,370 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ary_Arab.yaml' 2026-05-11T14:30:04,372 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-arz_Arab.yaml' 2026-05-11T14:30:04,373 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bam_Latn.yaml' 2026-05-11T14:30:04,374 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ban_Latn.yaml' 2026-05-11T14:30:04,375 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bem_Latn.yaml' 2026-05-11T14:30:04,376 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-cjk_Latn.yaml' 2026-05-11T14:30:04,377 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dik_Latn.yaml' 2026-05-11T14:30:04,378 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dyu_Latn.yaml' 2026-05-11T14:30:04,380 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ewe_Latn.yaml' 2026-05-11T14:30:04,381 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fon_Latn.yaml' 2026-05-11T14:30:04,382 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fra_Latn.yaml' 2026-05-11T14:30:04,383 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fuv_Latn.yaml' 2026-05-11T14:30:04,384 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-gaz_Latn.yaml' 2026-05-11T14:30:04,385 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-hau_Latn.yaml' 2026-05-11T14:30:04,387 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ibo_Latn.yaml' 2026-05-11T14:30:04,388 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kab_Latn.yaml' 2026-05-11T14:30:04,389 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kam_Latn.yaml' 2026-05-11T14:30:04,390 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kbp_Latn.yaml' 2026-05-11T14:30:04,391 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kea_Latn.yaml' 2026-05-11T14:30:04,392 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kik_Latn.yaml' 2026-05-11T14:30:04,394 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kin_Latn.yaml' 2026-05-11T14:30:04,395 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kmb_Latn.yaml' 2026-05-11T14:30:04,396 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Arab.yaml' 2026-05-11T14:30:04,397 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Latn.yaml' 2026-05-11T14:30:04,398 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kon_Latn.yaml' 2026-05-11T14:30:04,400 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lin_Latn.yaml' 2026-05-11T14:30:04,401 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lua_Latn.yaml' 2026-05-11T14:30:04,402 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lug_Latn.yaml' 2026-05-11T14:30:04,403 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-luo_Latn.yaml' 2026-05-11T14:30:04,404 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-mos_Latn.yaml' 2026-05-11T14:30:04,405 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nso_Latn.yaml' 2026-05-11T14:30:04,406 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nus_Latn.yaml' 2026-05-11T14:30:04,407 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nya_Latn.yaml' 2026-05-11T14:30:04,408 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-plt_Latn.yaml' 2026-05-11T14:30:04,409 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-run_Latn.yaml' 2026-05-11T14:30:04,410 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sag_Latn.yaml' 2026-05-11T14:30:04,412 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sna_Latn.yaml' 2026-05-11T14:30:04,413 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-som_Latn.yaml' 2026-05-11T14:30:04,414 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sot_Latn.yaml' 2026-05-11T14:30:04,415 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ssw_Latn.yaml' 2026-05-11T14:30:04,416 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sun_Latn.yaml' 2026-05-11T14:30:04,417 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-swh_Latn.yaml' 2026-05-11T14:30:04,418 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Latn.yaml' 2026-05-11T14:30:04,420 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Tfng.yaml' 2026-05-11T14:30:04,421 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tir_Ethi.yaml' 2026-05-11T14:30:04,422 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tsn_Latn.yaml' 2026-05-11T14:30:04,423 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tso_Latn.yaml' 2026-05-11T14:30:04,424 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tum_Latn.yaml' 2026-05-11T14:30:04,425 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-twi_Latn.yaml' 2026-05-11T14:30:04,427 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tzm_Tfng.yaml' 2026-05-11T14:30:04,428 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-umb_Latn.yaml' 2026-05-11T14:30:04,429 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-wol_Latn.yaml' 2026-05-11T14:30:04,430 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-xho_Latn.yaml' 2026-05-11T14:30:04,431 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-yor_Latn.yaml' 2026-05-11T14:30:04,432 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-zul_Latn.yaml' 2026-05-11T14:30:04,434 adding 'lm_eval/tasks/afrobench/flores/prompt_3/flores' 2026-05-11T14:30:04,437 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores' 2026-05-11T14:30:04,438 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,439 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,440 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_acq_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,442 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aeb_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,443 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_afr_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,444 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aka_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,445 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_amh_Ethi-eng_Latn.yaml' 2026-05-11T14:30:04,446 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ary_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,447 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_arz_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,448 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bam_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,449 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ban_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,450 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bem_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,451 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_cjk_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,453 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dik_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,454 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dyu_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,455 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ewe_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,456 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fon_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,458 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fra_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,459 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fuv_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,460 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_gaz_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,461 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_hau_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,462 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ibo_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,463 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kab_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,465 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kam_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,466 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kbp_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,467 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kea_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,468 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kik_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,469 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kin_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,470 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kmb_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,472 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Arab-eng_Latn.yaml' 2026-05-11T14:30:04,473 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,474 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kon_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,475 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lin_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,476 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lua_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,477 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lug_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,479 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_luo_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,480 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_mos_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,481 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nso_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,482 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nus_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,483 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nya_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,484 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_plt_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,485 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_run_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,486 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sag_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,487 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sna_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,488 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_som_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,490 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sot_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,491 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ssw_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,492 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sun_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,493 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_swh_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,494 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,495 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Tfng-eng_Latn.yaml' 2026-05-11T14:30:04,496 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tir_Ethi-eng_Latn.yaml' 2026-05-11T14:30:04,498 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tsn_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,499 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tso_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,500 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tum_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,501 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_twi_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,502 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tzm_Tfng-eng_Latn.yaml' 2026-05-11T14:30:04,503 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_umb_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,504 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_wol_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,505 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_xho_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,507 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_yor_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,508 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_zul_Latn-eng_Latn.yaml' 2026-05-11T14:30:04,511 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores' 2026-05-11T14:30:04,512 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Arab.yaml' 2026-05-11T14:30:04,513 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Latn.yaml' 2026-05-11T14:30:04,514 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-acq_Arab.yaml' 2026-05-11T14:30:04,515 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aeb_Arab.yaml' 2026-05-11T14:30:04,516 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-afr_Latn.yaml' 2026-05-11T14:30:04,518 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aka_Latn.yaml' 2026-05-11T14:30:04,519 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-amh_Ethi.yaml' 2026-05-11T14:30:04,520 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ary_Arab.yaml' 2026-05-11T14:30:04,521 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-arz_Arab.yaml' 2026-05-11T14:30:04,522 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bam_Latn.yaml' 2026-05-11T14:30:04,523 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ban_Latn.yaml' 2026-05-11T14:30:04,524 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bem_Latn.yaml' 2026-05-11T14:30:04,525 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-cjk_Latn.yaml' 2026-05-11T14:30:04,527 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dik_Latn.yaml' 2026-05-11T14:30:04,528 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dyu_Latn.yaml' 2026-05-11T14:30:04,529 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ewe_Latn.yaml' 2026-05-11T14:30:04,530 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fon_Latn.yaml' 2026-05-11T14:30:04,531 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fra_Latn.yaml' 2026-05-11T14:30:04,532 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fuv_Latn.yaml' 2026-05-11T14:30:04,533 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-gaz_Latn.yaml' 2026-05-11T14:30:04,534 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-hau_Latn.yaml' 2026-05-11T14:30:04,535 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ibo_Latn.yaml' 2026-05-11T14:30:04,537 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kab_Latn.yaml' 2026-05-11T14:30:04,538 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kam_Latn.yaml' 2026-05-11T14:30:04,539 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kbp_Latn.yaml' 2026-05-11T14:30:04,540 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kea_Latn.yaml' 2026-05-11T14:30:04,541 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kik_Latn.yaml' 2026-05-11T14:30:04,543 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kin_Latn.yaml' 2026-05-11T14:30:04,544 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kmb_Latn.yaml' 2026-05-11T14:30:04,545 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Arab.yaml' 2026-05-11T14:30:04,546 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Latn.yaml' 2026-05-11T14:30:04,548 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kon_Latn.yaml' 2026-05-11T14:30:04,549 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lin_Latn.yaml' 2026-05-11T14:30:04,550 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lua_Latn.yaml' 2026-05-11T14:30:04,551 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lug_Latn.yaml' 2026-05-11T14:30:04,553 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-luo_Latn.yaml' 2026-05-11T14:30:04,554 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-mos_Latn.yaml' 2026-05-11T14:30:04,555 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nso_Latn.yaml' 2026-05-11T14:30:04,556 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nus_Latn.yaml' 2026-05-11T14:30:04,558 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nya_Latn.yaml' 2026-05-11T14:30:04,559 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-plt_Latn.yaml' 2026-05-11T14:30:04,560 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-run_Latn.yaml' 2026-05-11T14:30:04,561 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sag_Latn.yaml' 2026-05-11T14:30:04,563 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sna_Latn.yaml' 2026-05-11T14:30:04,564 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-som_Latn.yaml' 2026-05-11T14:30:04,565 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sot_Latn.yaml' 2026-05-11T14:30:04,566 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ssw_Latn.yaml' 2026-05-11T14:30:04,568 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sun_Latn.yaml' 2026-05-11T14:30:04,569 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-swh_Latn.yaml' 2026-05-11T14:30:04,570 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Latn.yaml' 2026-05-11T14:30:04,571 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Tfng.yaml' 2026-05-11T14:30:04,573 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tir_Ethi.yaml' 2026-05-11T14:30:04,574 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tsn_Latn.yaml' 2026-05-11T14:30:04,575 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tso_Latn.yaml' 2026-05-11T14:30:04,577 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tum_Latn.yaml' 2026-05-11T14:30:04,578 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-twi_Latn.yaml' 2026-05-11T14:30:04,579 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tzm_Tfng.yaml' 2026-05-11T14:30:04,580 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-umb_Latn.yaml' 2026-05-11T14:30:04,582 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-wol_Latn.yaml' 2026-05-11T14:30:04,583 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-xho_Latn.yaml' 2026-05-11T14:30:04,584 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-yor_Latn.yaml' 2026-05-11T14:30:04,585 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-zul_Latn.yaml' 2026-05-11T14:30:04,587 adding 'lm_eval/tasks/afrobench/injongointent/README.md' 2026-05-11T14:30:04,589 adding 'lm_eval/tasks/afrobench/injongointent/gen_utils.py' 2026-05-11T14:30:04,590 adding 'lm_eval/tasks/afrobench/injongointent/injongointent.yaml' 2026-05-11T14:30:04,592 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent' 2026-05-11T14:30:04,594 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_amh.yaml' 2026-05-11T14:30:04,595 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_eng.yaml' 2026-05-11T14:30:04,596 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ewe.yaml' 2026-05-11T14:30:04,597 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_hau.yaml' 2026-05-11T14:30:04,599 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ibo.yaml' 2026-05-11T14:30:04,600 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_kin.yaml' 2026-05-11T14:30:04,601 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lin.yaml' 2026-05-11T14:30:04,602 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lug.yaml' 2026-05-11T14:30:04,604 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_orm.yaml' 2026-05-11T14:30:04,605 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sna.yaml' 2026-05-11T14:30:04,606 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sot.yaml' 2026-05-11T14:30:04,607 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_swa.yaml' 2026-05-11T14:30:04,609 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_twi.yaml' 2026-05-11T14:30:04,610 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_wol.yaml' 2026-05-11T14:30:04,611 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_xho.yaml' 2026-05-11T14:30:04,612 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_yor.yaml' 2026-05-11T14:30:04,613 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_zul.yaml' 2026-05-11T14:30:04,614 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/utils.py' 2026-05-11T14:30:04,616 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent' 2026-05-11T14:30:04,617 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_amh.yaml' 2026-05-11T14:30:04,618 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_eng.yaml' 2026-05-11T14:30:04,620 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ewe.yaml' 2026-05-11T14:30:04,621 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_hau.yaml' 2026-05-11T14:30:04,622 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ibo.yaml' 2026-05-11T14:30:04,623 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_kin.yaml' 2026-05-11T14:30:04,625 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lin.yaml' 2026-05-11T14:30:04,626 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lug.yaml' 2026-05-11T14:30:04,627 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_orm.yaml' 2026-05-11T14:30:04,628 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sna.yaml' 2026-05-11T14:30:04,629 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sot.yaml' 2026-05-11T14:30:04,630 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_swa.yaml' 2026-05-11T14:30:04,632 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_twi.yaml' 2026-05-11T14:30:04,633 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_wol.yaml' 2026-05-11T14:30:04,634 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_xho.yaml' 2026-05-11T14:30:04,635 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_yor.yaml' 2026-05-11T14:30:04,637 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_zul.yaml' 2026-05-11T14:30:04,638 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/utils.py' 2026-05-11T14:30:04,640 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent' 2026-05-11T14:30:04,641 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_amh.yaml' 2026-05-11T14:30:04,642 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_eng.yaml' 2026-05-11T14:30:04,643 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ewe.yaml' 2026-05-11T14:30:04,645 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_hau.yaml' 2026-05-11T14:30:04,646 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ibo.yaml' 2026-05-11T14:30:04,647 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_kin.yaml' 2026-05-11T14:30:04,648 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lin.yaml' 2026-05-11T14:30:04,650 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lug.yaml' 2026-05-11T14:30:04,651 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_orm.yaml' 2026-05-11T14:30:04,652 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sna.yaml' 2026-05-11T14:30:04,653 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sot.yaml' 2026-05-11T14:30:04,655 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_swa.yaml' 2026-05-11T14:30:04,656 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_twi.yaml' 2026-05-11T14:30:04,657 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_wol.yaml' 2026-05-11T14:30:04,658 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_xho.yaml' 2026-05-11T14:30:04,660 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_yor.yaml' 2026-05-11T14:30:04,661 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_zul.yaml' 2026-05-11T14:30:04,662 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/utils.py' 2026-05-11T14:30:04,664 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent' 2026-05-11T14:30:04,665 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_amh.yaml' 2026-05-11T14:30:04,666 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_eng.yaml' 2026-05-11T14:30:04,668 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ewe.yaml' 2026-05-11T14:30:04,669 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_hau.yaml' 2026-05-11T14:30:04,670 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ibo.yaml' 2026-05-11T14:30:04,671 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_kin.yaml' 2026-05-11T14:30:04,672 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lin.yaml' 2026-05-11T14:30:04,674 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lug.yaml' 2026-05-11T14:30:04,675 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_orm.yaml' 2026-05-11T14:30:04,676 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sna.yaml' 2026-05-11T14:30:04,677 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sot.yaml' 2026-05-11T14:30:04,679 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_swa.yaml' 2026-05-11T14:30:04,680 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_twi.yaml' 2026-05-11T14:30:04,681 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_wol.yaml' 2026-05-11T14:30:04,682 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_xho.yaml' 2026-05-11T14:30:04,684 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_yor.yaml' 2026-05-11T14:30:04,685 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_zul.yaml' 2026-05-11T14:30:04,686 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/utils.py' 2026-05-11T14:30:04,688 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent' 2026-05-11T14:30:04,689 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_amh.yaml' 2026-05-11T14:30:04,690 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_eng.yaml' 2026-05-11T14:30:04,692 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ewe.yaml' 2026-05-11T14:30:04,693 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_hau.yaml' 2026-05-11T14:30:04,694 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ibo.yaml' 2026-05-11T14:30:04,695 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_kin.yaml' 2026-05-11T14:30:04,697 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lin.yaml' 2026-05-11T14:30:04,698 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lug.yaml' 2026-05-11T14:30:04,699 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_orm.yaml' 2026-05-11T14:30:04,700 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sna.yaml' 2026-05-11T14:30:04,702 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sot.yaml' 2026-05-11T14:30:04,703 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_swa.yaml' 2026-05-11T14:30:04,704 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_twi.yaml' 2026-05-11T14:30:04,705 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_wol.yaml' 2026-05-11T14:30:04,707 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_xho.yaml' 2026-05-11T14:30:04,708 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_yor.yaml' 2026-05-11T14:30:04,709 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_zul.yaml' 2026-05-11T14:30:04,710 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/utils.py' 2026-05-11T14:30:04,712 adding 'lm_eval/tasks/afrobench/mafand/README.md' 2026-05-11T14:30:04,714 adding 'lm_eval/tasks/afrobench/mafand/gen_utils.py' 2026-05-11T14:30:04,715 adding 'lm_eval/tasks/afrobench/mafand/mafand.yaml' 2026-05-11T14:30:04,718 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand' 2026-05-11T14:30:04,719 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_amh-en.yaml' 2026-05-11T14:30:04,720 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bam-fr.yaml' 2026-05-11T14:30:04,721 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bbj-fr.yaml' 2026-05-11T14:30:04,722 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ewe-fr.yaml' 2026-05-11T14:30:04,723 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_fon-fr.yaml' 2026-05-11T14:30:04,725 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_hau-en.yaml' 2026-05-11T14:30:04,726 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ibo-en.yaml' 2026-05-11T14:30:04,727 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_kin-en.yaml' 2026-05-11T14:30:04,728 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_lug-en.yaml' 2026-05-11T14:30:04,729 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_luo-en.yaml' 2026-05-11T14:30:04,730 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_mos-fr.yaml' 2026-05-11T14:30:04,732 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_nya-en.yaml' 2026-05-11T14:30:04,733 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_pcm-en.yaml' 2026-05-11T14:30:04,734 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_sna-en.yaml' 2026-05-11T14:30:04,735 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_swa-en.yaml' 2026-05-11T14:30:04,736 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_tsn-en.yaml' 2026-05-11T14:30:04,737 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_twi-en.yaml' 2026-05-11T14:30:04,738 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_wol-fr.yaml' 2026-05-11T14:30:04,740 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_xho-en.yaml' 2026-05-11T14:30:04,741 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_yor-en.yaml' 2026-05-11T14:30:04,742 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_zul-en.yaml' 2026-05-11T14:30:04,743 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/utils.py' 2026-05-11T14:30:04,745 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand' 2026-05-11T14:30:04,747 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-amh.yaml' 2026-05-11T14:30:04,748 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-hau.yaml' 2026-05-11T14:30:04,749 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-ibo.yaml' 2026-05-11T14:30:04,750 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-kin.yaml' 2026-05-11T14:30:04,751 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-lug.yaml' 2026-05-11T14:30:04,752 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-luo.yaml' 2026-05-11T14:30:04,753 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-nya.yaml' 2026-05-11T14:30:04,755 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-pcm.yaml' 2026-05-11T14:30:04,756 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-sna.yaml' 2026-05-11T14:30:04,757 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-swa.yaml' 2026-05-11T14:30:04,758 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-tsn.yaml' 2026-05-11T14:30:04,759 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-twi.yaml' 2026-05-11T14:30:04,760 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-xho.yaml' 2026-05-11T14:30:04,761 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-yor.yaml' 2026-05-11T14:30:04,762 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-zul.yaml' 2026-05-11T14:30:04,763 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bam.yaml' 2026-05-11T14:30:04,764 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bbj.yaml' 2026-05-11T14:30:04,765 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-ewe.yaml' 2026-05-11T14:30:04,766 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-fon.yaml' 2026-05-11T14:30:04,767 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-mos.yaml' 2026-05-11T14:30:04,769 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-wol.yaml' 2026-05-11T14:30:04,770 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/utils.py' 2026-05-11T14:30:04,772 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand' 2026-05-11T14:30:04,773 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_amh-en.yaml' 2026-05-11T14:30:04,775 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bam-fr.yaml' 2026-05-11T14:30:04,776 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bbj-fr.yaml' 2026-05-11T14:30:04,777 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ewe-fr.yaml' 2026-05-11T14:30:04,778 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_fon-fr.yaml' 2026-05-11T14:30:04,779 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_hau-en.yaml' 2026-05-11T14:30:04,781 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ibo-en.yaml' 2026-05-11T14:30:04,782 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_kin-en.yaml' 2026-05-11T14:30:04,783 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_lug-en.yaml' 2026-05-11T14:30:04,784 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_luo-en.yaml' 2026-05-11T14:30:04,785 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_mos-fr.yaml' 2026-05-11T14:30:04,786 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_nya-en.yaml' 2026-05-11T14:30:04,787 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_pcm-en.yaml' 2026-05-11T14:30:04,788 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_sna-en.yaml' 2026-05-11T14:30:04,790 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_swa-en.yaml' 2026-05-11T14:30:04,791 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_tsn-en.yaml' 2026-05-11T14:30:04,792 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_twi-en.yaml' 2026-05-11T14:30:04,793 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_wol-fr.yaml' 2026-05-11T14:30:04,794 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_xho-en.yaml' 2026-05-11T14:30:04,795 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_yor-en.yaml' 2026-05-11T14:30:04,797 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_zul-en.yaml' 2026-05-11T14:30:04,798 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/utils.py' 2026-05-11T14:30:04,800 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand' 2026-05-11T14:30:04,801 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-amh.yaml' 2026-05-11T14:30:04,802 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-hau.yaml' 2026-05-11T14:30:04,803 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-ibo.yaml' 2026-05-11T14:30:04,804 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-kin.yaml' 2026-05-11T14:30:04,805 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-lug.yaml' 2026-05-11T14:30:04,806 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-luo.yaml' 2026-05-11T14:30:04,807 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-nya.yaml' 2026-05-11T14:30:04,809 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-pcm.yaml' 2026-05-11T14:30:04,810 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-sna.yaml' 2026-05-11T14:30:04,811 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-swa.yaml' 2026-05-11T14:30:04,812 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-tsn.yaml' 2026-05-11T14:30:04,813 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-twi.yaml' 2026-05-11T14:30:04,814 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-xho.yaml' 2026-05-11T14:30:04,815 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-yor.yaml' 2026-05-11T14:30:04,816 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-zul.yaml' 2026-05-11T14:30:04,818 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bam.yaml' 2026-05-11T14:30:04,819 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bbj.yaml' 2026-05-11T14:30:04,820 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-ewe.yaml' 2026-05-11T14:30:04,821 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-fon.yaml' 2026-05-11T14:30:04,822 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-mos.yaml' 2026-05-11T14:30:04,823 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-wol.yaml' 2026-05-11T14:30:04,824 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/utils.py' 2026-05-11T14:30:04,827 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand' 2026-05-11T14:30:04,828 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_amh-en.yaml' 2026-05-11T14:30:04,829 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bam-fr.yaml' 2026-05-11T14:30:04,830 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bbj-fr.yaml' 2026-05-11T14:30:04,832 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ewe-fr.yaml' 2026-05-11T14:30:04,833 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_fon-fr.yaml' 2026-05-11T14:30:04,834 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_hau-en.yaml' 2026-05-11T14:30:04,835 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ibo-en.yaml' 2026-05-11T14:30:04,836 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_kin-en.yaml' 2026-05-11T14:30:04,838 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_lug-en.yaml' 2026-05-11T14:30:04,839 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_luo-en.yaml' 2026-05-11T14:30:04,840 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_mos-fr.yaml' 2026-05-11T14:30:04,841 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_nya-en.yaml' 2026-05-11T14:30:04,842 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_pcm-en.yaml' 2026-05-11T14:30:04,844 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_sna-en.yaml' 2026-05-11T14:30:04,845 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_swa-en.yaml' 2026-05-11T14:30:04,846 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_tsn-en.yaml' 2026-05-11T14:30:04,847 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_twi-en.yaml' 2026-05-11T14:30:04,848 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_wol-fr.yaml' 2026-05-11T14:30:04,849 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_xho-en.yaml' 2026-05-11T14:30:04,850 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_yor-en.yaml' 2026-05-11T14:30:04,851 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_zul-en.yaml' 2026-05-11T14:30:04,853 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/utils.py' 2026-05-11T14:30:04,855 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand' 2026-05-11T14:30:04,856 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-amh.yaml' 2026-05-11T14:30:04,857 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-hau.yaml' 2026-05-11T14:30:04,858 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-ibo.yaml' 2026-05-11T14:30:04,859 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-kin.yaml' 2026-05-11T14:30:04,860 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-lug.yaml' 2026-05-11T14:30:04,862 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-luo.yaml' 2026-05-11T14:30:04,863 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-nya.yaml' 2026-05-11T14:30:04,864 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-pcm.yaml' 2026-05-11T14:30:04,865 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-sna.yaml' 2026-05-11T14:30:04,866 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-swa.yaml' 2026-05-11T14:30:04,868 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-tsn.yaml' 2026-05-11T14:30:04,869 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-twi.yaml' 2026-05-11T14:30:04,870 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-xho.yaml' 2026-05-11T14:30:04,871 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-yor.yaml' 2026-05-11T14:30:04,872 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-zul.yaml' 2026-05-11T14:30:04,873 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bam.yaml' 2026-05-11T14:30:04,874 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bbj.yaml' 2026-05-11T14:30:04,876 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-ewe.yaml' 2026-05-11T14:30:04,877 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-fon.yaml' 2026-05-11T14:30:04,878 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-mos.yaml' 2026-05-11T14:30:04,879 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-wol.yaml' 2026-05-11T14:30:04,881 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/utils.py' 2026-05-11T14:30:04,883 adding 'lm_eval/tasks/afrobench/masakhaner/README.md' 2026-05-11T14:30:04,884 adding 'lm_eval/tasks/afrobench/masakhaner/gen_utils.py' 2026-05-11T14:30:04,885 adding 'lm_eval/tasks/afrobench/masakhaner/masakhaner.yaml' 2026-05-11T14:30:04,887 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner' 2026-05-11T14:30:04,889 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_am.yaml' 2026-05-11T14:30:04,890 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bbj.yaml' 2026-05-11T14:30:04,891 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bm.yaml' 2026-05-11T14:30:04,892 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ee.yaml' 2026-05-11T14:30:04,894 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ha.yaml' 2026-05-11T14:30:04,895 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ig.yaml' 2026-05-11T14:30:04,896 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_lg.yaml' 2026-05-11T14:30:04,898 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_luo.yaml' 2026-05-11T14:30:04,899 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_mos.yaml' 2026-05-11T14:30:04,900 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ny.yaml' 2026-05-11T14:30:04,901 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_pcm.yaml' 2026-05-11T14:30:04,903 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_rw.yaml' 2026-05-11T14:30:04,904 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sn.yaml' 2026-05-11T14:30:04,905 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sw.yaml' 2026-05-11T14:30:04,906 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tn.yaml' 2026-05-11T14:30:04,907 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tw.yaml' 2026-05-11T14:30:04,909 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_wo.yaml' 2026-05-11T14:30:04,910 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_xh.yaml' 2026-05-11T14:30:04,911 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_yo.yaml' 2026-05-11T14:30:04,912 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_zu.yaml' 2026-05-11T14:30:04,914 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/utils.py' 2026-05-11T14:30:04,916 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner' 2026-05-11T14:30:04,917 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_am.yaml' 2026-05-11T14:30:04,918 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bbj.yaml' 2026-05-11T14:30:04,919 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bm.yaml' 2026-05-11T14:30:04,921 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ee.yaml' 2026-05-11T14:30:04,922 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ha.yaml' 2026-05-11T14:30:04,923 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ig.yaml' 2026-05-11T14:30:04,924 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_lg.yaml' 2026-05-11T14:30:04,926 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_luo.yaml' 2026-05-11T14:30:04,927 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_mos.yaml' 2026-05-11T14:30:04,928 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ny.yaml' 2026-05-11T14:30:04,929 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_pcm.yaml' 2026-05-11T14:30:04,931 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_rw.yaml' 2026-05-11T14:30:04,932 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sn.yaml' 2026-05-11T14:30:04,933 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sw.yaml' 2026-05-11T14:30:04,934 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tn.yaml' 2026-05-11T14:30:04,936 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tw.yaml' 2026-05-11T14:30:04,937 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_wo.yaml' 2026-05-11T14:30:04,938 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_xh.yaml' 2026-05-11T14:30:04,940 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_yo.yaml' 2026-05-11T14:30:04,941 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_zu.yaml' 2026-05-11T14:30:04,943 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/utils.py' 2026-05-11T14:30:04,945 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner' 2026-05-11T14:30:04,946 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_am.yaml' 2026-05-11T14:30:04,947 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bbj.yaml' 2026-05-11T14:30:04,949 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bm.yaml' 2026-05-11T14:30:04,950 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ee.yaml' 2026-05-11T14:30:04,951 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ha.yaml' 2026-05-11T14:30:04,952 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ig.yaml' 2026-05-11T14:30:04,953 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_lg.yaml' 2026-05-11T14:30:04,954 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_luo.yaml' 2026-05-11T14:30:04,956 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_mos.yaml' 2026-05-11T14:30:04,957 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ny.yaml' 2026-05-11T14:30:04,958 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_pcm.yaml' 2026-05-11T14:30:04,959 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_rw.yaml' 2026-05-11T14:30:04,960 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sn.yaml' 2026-05-11T14:30:04,961 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sw.yaml' 2026-05-11T14:30:04,962 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tn.yaml' 2026-05-11T14:30:04,964 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tw.yaml' 2026-05-11T14:30:04,965 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_wo.yaml' 2026-05-11T14:30:04,966 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_xh.yaml' 2026-05-11T14:30:04,967 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_yo.yaml' 2026-05-11T14:30:04,968 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_zu.yaml' 2026-05-11T14:30:04,970 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/utils.py' 2026-05-11T14:30:04,972 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner' 2026-05-11T14:30:04,973 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_am.yaml' 2026-05-11T14:30:04,974 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bbj.yaml' 2026-05-11T14:30:04,975 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bm.yaml' 2026-05-11T14:30:04,976 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ee.yaml' 2026-05-11T14:30:04,978 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ha.yaml' 2026-05-11T14:30:04,979 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ig.yaml' 2026-05-11T14:30:04,980 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_lg.yaml' 2026-05-11T14:30:04,981 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_luo.yaml' 2026-05-11T14:30:04,982 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_mos.yaml' 2026-05-11T14:30:04,983 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ny.yaml' 2026-05-11T14:30:04,985 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_pcm.yaml' 2026-05-11T14:30:04,986 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_rw.yaml' 2026-05-11T14:30:04,987 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sn.yaml' 2026-05-11T14:30:04,988 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sw.yaml' 2026-05-11T14:30:04,990 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tn.yaml' 2026-05-11T14:30:04,991 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tw.yaml' 2026-05-11T14:30:04,992 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_wo.yaml' 2026-05-11T14:30:04,994 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_xh.yaml' 2026-05-11T14:30:04,995 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_yo.yaml' 2026-05-11T14:30:04,996 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_zu.yaml' 2026-05-11T14:30:04,997 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/utils.py' 2026-05-11T14:30:04,999 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner' 2026-05-11T14:30:05,001 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_am.yaml' 2026-05-11T14:30:05,002 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bbj.yaml' 2026-05-11T14:30:05,003 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bm.yaml' 2026-05-11T14:30:05,004 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ee.yaml' 2026-05-11T14:30:05,005 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ha.yaml' 2026-05-11T14:30:05,006 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ig.yaml' 2026-05-11T14:30:05,007 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_lg.yaml' 2026-05-11T14:30:05,009 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_luo.yaml' 2026-05-11T14:30:05,010 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_mos.yaml' 2026-05-11T14:30:05,011 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ny.yaml' 2026-05-11T14:30:05,012 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_pcm.yaml' 2026-05-11T14:30:05,013 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_rw.yaml' 2026-05-11T14:30:05,014 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sn.yaml' 2026-05-11T14:30:05,016 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sw.yaml' 2026-05-11T14:30:05,017 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tn.yaml' 2026-05-11T14:30:05,018 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tw.yaml' 2026-05-11T14:30:05,019 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_wo.yaml' 2026-05-11T14:30:05,021 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_xh.yaml' 2026-05-11T14:30:05,022 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_yo.yaml' 2026-05-11T14:30:05,023 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_zu.yaml' 2026-05-11T14:30:05,024 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/utils.py' 2026-05-11T14:30:05,027 adding 'lm_eval/tasks/afrobench/masakhanews/README.md' 2026-05-11T14:30:05,028 adding 'lm_eval/tasks/afrobench/masakhanews/masakhanews.yaml' 2026-05-11T14:30:05,029 adding 'lm_eval/tasks/afrobench/masakhanews/utils.py' 2026-05-11T14:30:05,031 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews' 2026-05-11T14:30:05,032 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_amh.yaml' 2026-05-11T14:30:05,034 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_eng.yaml' 2026-05-11T14:30:05,035 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_fra.yaml' 2026-05-11T14:30:05,036 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_hau.yaml' 2026-05-11T14:30:05,037 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_ibo.yaml' 2026-05-11T14:30:05,038 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lin.yaml' 2026-05-11T14:30:05,039 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lug.yaml' 2026-05-11T14:30:05,041 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_orm.yaml' 2026-05-11T14:30:05,042 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_pcm.yaml' 2026-05-11T14:30:05,043 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_run.yaml' 2026-05-11T14:30:05,044 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_sna.yaml' 2026-05-11T14:30:05,045 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_som.yaml' 2026-05-11T14:30:05,046 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_swa.yaml' 2026-05-11T14:30:05,048 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_tir.yaml' 2026-05-11T14:30:05,049 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_xho.yaml' 2026-05-11T14:30:05,050 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_yor.yaml' 2026-05-11T14:30:05,051 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/utils.py' 2026-05-11T14:30:05,053 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews' 2026-05-11T14:30:05,054 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_amh.yaml' 2026-05-11T14:30:05,056 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_eng.yaml' 2026-05-11T14:30:05,057 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_fra.yaml' 2026-05-11T14:30:05,058 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_hau.yaml' 2026-05-11T14:30:05,059 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_ibo.yaml' 2026-05-11T14:30:05,061 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lin.yaml' 2026-05-11T14:30:05,062 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lug.yaml' 2026-05-11T14:30:05,063 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_orm.yaml' 2026-05-11T14:30:05,064 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_pcm.yaml' 2026-05-11T14:30:05,065 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_run.yaml' 2026-05-11T14:30:05,066 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_sna.yaml' 2026-05-11T14:30:05,068 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_som.yaml' 2026-05-11T14:30:05,069 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_swa.yaml' 2026-05-11T14:30:05,070 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_tir.yaml' 2026-05-11T14:30:05,071 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_xho.yaml' 2026-05-11T14:30:05,072 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_yor.yaml' 2026-05-11T14:30:05,073 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/utils.py' 2026-05-11T14:30:05,075 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews' 2026-05-11T14:30:05,077 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_amh.yaml' 2026-05-11T14:30:05,078 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_eng.yaml' 2026-05-11T14:30:05,079 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_fra.yaml' 2026-05-11T14:30:05,080 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_hau.yaml' 2026-05-11T14:30:05,081 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_ibo.yaml' 2026-05-11T14:30:05,083 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lin.yaml' 2026-05-11T14:30:05,084 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lug.yaml' 2026-05-11T14:30:05,085 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_orm.yaml' 2026-05-11T14:30:05,086 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_pcm.yaml' 2026-05-11T14:30:05,087 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_run.yaml' 2026-05-11T14:30:05,089 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_sna.yaml' 2026-05-11T14:30:05,090 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_som.yaml' 2026-05-11T14:30:05,091 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_swa.yaml' 2026-05-11T14:30:05,092 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_tir.yaml' 2026-05-11T14:30:05,093 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_xho.yaml' 2026-05-11T14:30:05,095 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_yor.yaml' 2026-05-11T14:30:05,096 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/utils.py' 2026-05-11T14:30:05,098 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews' 2026-05-11T14:30:05,099 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_amh.yaml' 2026-05-11T14:30:05,100 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_eng.yaml' 2026-05-11T14:30:05,102 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_fra.yaml' 2026-05-11T14:30:05,103 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_hau.yaml' 2026-05-11T14:30:05,104 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_ibo.yaml' 2026-05-11T14:30:05,105 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lin.yaml' 2026-05-11T14:30:05,106 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lug.yaml' 2026-05-11T14:30:05,107 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_orm.yaml' 2026-05-11T14:30:05,109 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_pcm.yaml' 2026-05-11T14:30:05,110 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_run.yaml' 2026-05-11T14:30:05,111 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_sna.yaml' 2026-05-11T14:30:05,112 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_som.yaml' 2026-05-11T14:30:05,113 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_swa.yaml' 2026-05-11T14:30:05,114 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_tir.yaml' 2026-05-11T14:30:05,115 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_xho.yaml' 2026-05-11T14:30:05,117 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_yor.yaml' 2026-05-11T14:30:05,118 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/utils.py' 2026-05-11T14:30:05,120 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews' 2026-05-11T14:30:05,121 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_amh.yaml' 2026-05-11T14:30:05,122 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_eng.yaml' 2026-05-11T14:30:05,123 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_fra.yaml' 2026-05-11T14:30:05,125 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_hau.yaml' 2026-05-11T14:30:05,126 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_ibo.yaml' 2026-05-11T14:30:05,127 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lin.yaml' 2026-05-11T14:30:05,129 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lug.yaml' 2026-05-11T14:30:05,130 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_orm.yaml' 2026-05-11T14:30:05,131 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_pcm.yaml' 2026-05-11T14:30:05,133 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_run.yaml' 2026-05-11T14:30:05,134 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_sna.yaml' 2026-05-11T14:30:05,135 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_som.yaml' 2026-05-11T14:30:05,137 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_swa.yaml' 2026-05-11T14:30:05,138 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_tir.yaml' 2026-05-11T14:30:05,139 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_xho.yaml' 2026-05-11T14:30:05,141 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_yor.yaml' 2026-05-11T14:30:05,142 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/utils.py' 2026-05-11T14:30:05,144 adding 'lm_eval/tasks/afrobench/masakhapos/README.md' 2026-05-11T14:30:05,145 adding 'lm_eval/tasks/afrobench/masakhapos/gen_utils.py' 2026-05-11T14:30:05,147 adding 'lm_eval/tasks/afrobench/masakhapos/masakhapos.yaml' 2026-05-11T14:30:05,148 adding 'lm_eval/tasks/afrobench/masakhapos/utils.py' 2026-05-11T14:30:05,150 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bam.yaml' 2026-05-11T14:30:05,151 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bbj.yaml' 2026-05-11T14:30:05,153 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ewe.yaml' 2026-05-11T14:30:05,154 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_fon.yaml' 2026-05-11T14:30:05,156 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_hau.yaml' 2026-05-11T14:30:05,157 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ibo.yaml' 2026-05-11T14:30:05,158 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_kin.yaml' 2026-05-11T14:30:05,160 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_lug.yaml' 2026-05-11T14:30:05,161 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_luo.yaml' 2026-05-11T14:30:05,163 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_mos.yaml' 2026-05-11T14:30:05,164 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_nya.yaml' 2026-05-11T14:30:05,166 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_pcm.yaml' 2026-05-11T14:30:05,167 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_sna.yaml' 2026-05-11T14:30:05,168 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_swa.yaml' 2026-05-11T14:30:05,170 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_tsn.yaml' 2026-05-11T14:30:05,171 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_twi.yaml' 2026-05-11T14:30:05,172 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_wol.yaml' 2026-05-11T14:30:05,174 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_xho.yaml' 2026-05-11T14:30:05,175 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yaml' 2026-05-11T14:30:05,178 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yor.yaml' 2026-05-11T14:30:05,180 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_zul.yaml' 2026-05-11T14:30:05,182 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/utils.py' 2026-05-11T14:30:05,185 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bam.yaml' 2026-05-11T14:30:05,186 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bbj.yaml' 2026-05-11T14:30:05,187 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ewe.yaml' 2026-05-11T14:30:05,189 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_fon.yaml' 2026-05-11T14:30:05,190 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_hau.yaml' 2026-05-11T14:30:05,191 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ibo.yaml' 2026-05-11T14:30:05,193 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_kin.yaml' 2026-05-11T14:30:05,194 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_lug.yaml' 2026-05-11T14:30:05,195 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_luo.yaml' 2026-05-11T14:30:05,197 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_mos.yaml' 2026-05-11T14:30:05,198 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_nya.yaml' 2026-05-11T14:30:05,199 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_pcm.yaml' 2026-05-11T14:30:05,201 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_sna.yaml' 2026-05-11T14:30:05,202 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_swa.yaml' 2026-05-11T14:30:05,203 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_tsn.yaml' 2026-05-11T14:30:05,204 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_twi.yaml' 2026-05-11T14:30:05,206 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_wol.yaml' 2026-05-11T14:30:05,207 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_xho.yaml' 2026-05-11T14:30:05,208 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yaml' 2026-05-11T14:30:05,209 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yor.yaml' 2026-05-11T14:30:05,211 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_zul.yaml' 2026-05-11T14:30:05,212 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/utils.py' 2026-05-11T14:30:05,214 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bam.yaml' 2026-05-11T14:30:05,215 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bbj.yaml' 2026-05-11T14:30:05,216 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ewe.yaml' 2026-05-11T14:30:05,218 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_fon.yaml' 2026-05-11T14:30:05,219 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_hau.yaml' 2026-05-11T14:30:05,220 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ibo.yaml' 2026-05-11T14:30:05,222 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_kin.yaml' 2026-05-11T14:30:05,223 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_lug.yaml' 2026-05-11T14:30:05,224 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_luo.yaml' 2026-05-11T14:30:05,225 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_mos.yaml' 2026-05-11T14:30:05,226 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_nya.yaml' 2026-05-11T14:30:05,227 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_pcm.yaml' 2026-05-11T14:30:05,228 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_sna.yaml' 2026-05-11T14:30:05,230 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_swa.yaml' 2026-05-11T14:30:05,231 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_tsn.yaml' 2026-05-11T14:30:05,232 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_twi.yaml' 2026-05-11T14:30:05,233 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_wol.yaml' 2026-05-11T14:30:05,235 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_xho.yaml' 2026-05-11T14:30:05,236 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yaml' 2026-05-11T14:30:05,237 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yor.yaml' 2026-05-11T14:30:05,238 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_zul.yaml' 2026-05-11T14:30:05,240 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/utils.py' 2026-05-11T14:30:05,242 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bam.yaml' 2026-05-11T14:30:05,243 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bbj.yaml' 2026-05-11T14:30:05,244 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ewe.yaml' 2026-05-11T14:30:05,245 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_fon.yaml' 2026-05-11T14:30:05,247 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_hau.yaml' 2026-05-11T14:30:05,248 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ibo.yaml' 2026-05-11T14:30:05,249 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_kin.yaml' 2026-05-11T14:30:05,250 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_lug.yaml' 2026-05-11T14:30:05,252 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_luo.yaml' 2026-05-11T14:30:05,253 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_mos.yaml' 2026-05-11T14:30:05,254 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_nya.yaml' 2026-05-11T14:30:05,255 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_pcm.yaml' 2026-05-11T14:30:05,257 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_sna.yaml' 2026-05-11T14:30:05,258 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_swa.yaml' 2026-05-11T14:30:05,259 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_tsn.yaml' 2026-05-11T14:30:05,261 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_twi.yaml' 2026-05-11T14:30:05,262 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_wol.yaml' 2026-05-11T14:30:05,264 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_xho.yaml' 2026-05-11T14:30:05,266 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yaml' 2026-05-11T14:30:05,268 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yor.yaml' 2026-05-11T14:30:05,270 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_zul.yaml' 2026-05-11T14:30:05,272 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/utils.py' 2026-05-11T14:30:05,274 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bam.yaml' 2026-05-11T14:30:05,275 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bbj.yaml' 2026-05-11T14:30:05,277 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ewe.yaml' 2026-05-11T14:30:05,278 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_fon.yaml' 2026-05-11T14:30:05,279 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_hau.yaml' 2026-05-11T14:30:05,280 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ibo.yaml' 2026-05-11T14:30:05,282 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_kin.yaml' 2026-05-11T14:30:05,283 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_lug.yaml' 2026-05-11T14:30:05,284 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_luo.yaml' 2026-05-11T14:30:05,285 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_mos.yaml' 2026-05-11T14:30:05,286 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_nya.yaml' 2026-05-11T14:30:05,287 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_pcm.yaml' 2026-05-11T14:30:05,289 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_sna.yaml' 2026-05-11T14:30:05,290 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_swa.yaml' 2026-05-11T14:30:05,291 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_tsn.yaml' 2026-05-11T14:30:05,292 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_twi.yaml' 2026-05-11T14:30:05,294 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_wol.yaml' 2026-05-11T14:30:05,295 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_xho.yaml' 2026-05-11T14:30:05,296 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yaml' 2026-05-11T14:30:05,297 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yor.yaml' 2026-05-11T14:30:05,298 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_zul.yaml' 2026-05-11T14:30:05,300 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/utils.py' 2026-05-11T14:30:05,302 adding 'lm_eval/tasks/afrobench/naijarc/README.md' 2026-05-11T14:30:05,303 adding 'lm_eval/tasks/afrobench/naijarc/naijarc.yaml' 2026-05-11T14:30:05,304 adding 'lm_eval/tasks/afrobench/naijarc/utils.py' 2026-05-11T14:30:05,306 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc' 2026-05-11T14:30:05,307 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_hau.yaml' 2026-05-11T14:30:05,309 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_ibo.yaml' 2026-05-11T14:30:05,310 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_yor.yaml' 2026-05-11T14:30:05,312 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc' 2026-05-11T14:30:05,313 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_hau.yaml' 2026-05-11T14:30:05,314 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_ibo.yaml' 2026-05-11T14:30:05,316 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_yor.yaml' 2026-05-11T14:30:05,318 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc' 2026-05-11T14:30:05,319 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_hau.yaml' 2026-05-11T14:30:05,320 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_ibo.yaml' 2026-05-11T14:30:05,321 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_yor.yaml' 2026-05-11T14:30:05,323 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc' 2026-05-11T14:30:05,324 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_hau.yaml' 2026-05-11T14:30:05,326 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_ibo.yaml' 2026-05-11T14:30:05,327 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_yor.yaml' 2026-05-11T14:30:05,329 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc' 2026-05-11T14:30:05,330 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_hau.yaml' 2026-05-11T14:30:05,331 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_ibo.yaml' 2026-05-11T14:30:05,333 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_yor.yaml' 2026-05-11T14:30:05,335 adding 'lm_eval/tasks/afrobench/nollysenti/README.md' 2026-05-11T14:30:05,336 adding 'lm_eval/tasks/afrobench/nollysenti/nollysenti.yaml' 2026-05-11T14:30:05,338 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti' 2026-05-11T14:30:05,339 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_eng.yaml' 2026-05-11T14:30:05,340 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_hau.yaml' 2026-05-11T14:30:05,341 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_ibo.yaml' 2026-05-11T14:30:05,342 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_pcm.yaml' 2026-05-11T14:30:05,343 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_yor.yaml' 2026-05-11T14:30:05,344 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/utils.py' 2026-05-11T14:30:05,346 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti' 2026-05-11T14:30:05,347 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_eng.yaml' 2026-05-11T14:30:05,348 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_hau.yaml' 2026-05-11T14:30:05,350 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_ibo.yaml' 2026-05-11T14:30:05,351 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_pcm.yaml' 2026-05-11T14:30:05,352 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_yor.yaml' 2026-05-11T14:30:05,353 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/utils.py' 2026-05-11T14:30:05,355 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti' 2026-05-11T14:30:05,356 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_eng.yaml' 2026-05-11T14:30:05,357 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_hau.yaml' 2026-05-11T14:30:05,358 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_ibo.yaml' 2026-05-11T14:30:05,360 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_pcm.yaml' 2026-05-11T14:30:05,361 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_yor.yaml' 2026-05-11T14:30:05,362 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/utils.py' 2026-05-11T14:30:05,364 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti' 2026-05-11T14:30:05,365 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_eng.yaml' 2026-05-11T14:30:05,366 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_hau.yaml' 2026-05-11T14:30:05,367 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_ibo.yaml' 2026-05-11T14:30:05,368 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_pcm.yaml' 2026-05-11T14:30:05,370 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_yor.yaml' 2026-05-11T14:30:05,371 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/utils.py' 2026-05-11T14:30:05,372 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti' 2026-05-11T14:30:05,373 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_eng.yaml' 2026-05-11T14:30:05,375 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_hau.yaml' 2026-05-11T14:30:05,376 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_ibo.yaml' 2026-05-11T14:30:05,377 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_pcm.yaml' 2026-05-11T14:30:05,379 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_yor.yaml' 2026-05-11T14:30:05,380 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/utils.py' 2026-05-11T14:30:05,382 adding 'lm_eval/tasks/afrobench/ntrex/README.md' 2026-05-11T14:30:05,383 adding 'lm_eval/tasks/afrobench/ntrex/gen_utils.py' 2026-05-11T14:30:05,384 adding 'lm_eval/tasks/afrobench/ntrex/ntrex.yaml' 2026-05-11T14:30:05,387 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex' 2026-05-11T14:30:05,388 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_afr_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,390 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_amh_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,391 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_arb_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,392 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_bem_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,394 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ewe_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,395 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_fra_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,396 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_hau_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,397 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ibo_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,398 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_kin_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,400 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mey_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,401 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mlg_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,402 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_msa_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,403 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nde_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,404 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nso_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,406 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nya_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,407 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_orm_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,408 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_shi_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,409 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_sna_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,410 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_som_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,412 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ssw_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,413 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_swa_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,414 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tam_Taml-eng_Latn.yaml' 2026-05-11T14:30:05,415 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tel_Telu-eng_Latn.yaml' 2026-05-11T14:30:05,416 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tir_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,418 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ton_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,419 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tsn_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,420 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_urd_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,421 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ven_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,422 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_wol_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,424 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_xho_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,425 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_yor_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,426 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_zul_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,429 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex' 2026-05-11T14:30:05,430 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-afr_Latn.yaml' 2026-05-11T14:30:05,431 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-amh_Ethi.yaml' 2026-05-11T14:30:05,433 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-arb_Arab.yaml' 2026-05-11T14:30:05,434 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-bem_Latn.yaml' 2026-05-11T14:30:05,435 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ewe_Latn.yaml' 2026-05-11T14:30:05,436 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-fra_Latn.yaml' 2026-05-11T14:30:05,437 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-hau_Latn.yaml' 2026-05-11T14:30:05,439 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ibo_Latn.yaml' 2026-05-11T14:30:05,440 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-kin_Latn.yaml' 2026-05-11T14:30:05,441 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mey_Arab.yaml' 2026-05-11T14:30:05,442 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mlg_Latn.yaml' 2026-05-11T14:30:05,443 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-msa_Latn.yaml' 2026-05-11T14:30:05,444 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nde_Latn.yaml' 2026-05-11T14:30:05,445 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nso_Latn.yaml' 2026-05-11T14:30:05,447 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nya_Latn.yaml' 2026-05-11T14:30:05,448 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-orm_Ethi.yaml' 2026-05-11T14:30:05,449 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-shi_Arab.yaml' 2026-05-11T14:30:05,450 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-sna_Latn.yaml' 2026-05-11T14:30:05,451 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-som_Latn.yaml' 2026-05-11T14:30:05,452 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ssw_Latn.yaml' 2026-05-11T14:30:05,454 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-swa_Latn.yaml' 2026-05-11T14:30:05,455 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tam_Taml.yaml' 2026-05-11T14:30:05,456 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tel_Telu.yaml' 2026-05-11T14:30:05,457 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tir_Ethi.yaml' 2026-05-11T14:30:05,459 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ton_Latn.yaml' 2026-05-11T14:30:05,460 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tsn_Latn.yaml' 2026-05-11T14:30:05,461 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-urd_Arab.yaml' 2026-05-11T14:30:05,462 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ven_Latn.yaml' 2026-05-11T14:30:05,463 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-wol_Latn.yaml' 2026-05-11T14:30:05,464 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-xho_Latn.yaml' 2026-05-11T14:30:05,466 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-yor_Latn.yaml' 2026-05-11T14:30:05,467 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-zul_Latn.yaml' 2026-05-11T14:30:05,470 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex' 2026-05-11T14:30:05,471 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_afr_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,472 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_amh_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,473 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_arb_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,474 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_bem_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,476 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ewe_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,477 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_fra_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,478 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_hau_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,480 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ibo_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,481 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_kin_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,482 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mey_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,483 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mlg_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,484 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_msa_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,485 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nde_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,487 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nso_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,488 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nya_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,489 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_orm_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,491 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_shi_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,492 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_sna_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,493 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_som_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,494 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ssw_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,496 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_swa_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,497 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tam_Taml-eng_Latn.yaml' 2026-05-11T14:30:05,498 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tel_Telu-eng_Latn.yaml' 2026-05-11T14:30:05,499 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tir_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,501 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ton_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,502 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tsn_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,503 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_urd_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,504 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ven_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,505 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_wol_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,506 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_xho_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,508 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_yor_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,509 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_zul_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,511 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex' 2026-05-11T14:30:05,512 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-afr_Latn.yaml' 2026-05-11T14:30:05,513 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-amh_Ethi.yaml' 2026-05-11T14:30:05,514 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-arb_Arab.yaml' 2026-05-11T14:30:05,516 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-bem_Latn.yaml' 2026-05-11T14:30:05,517 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ewe_Latn.yaml' 2026-05-11T14:30:05,518 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-fra_Latn.yaml' 2026-05-11T14:30:05,519 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-hau_Latn.yaml' 2026-05-11T14:30:05,521 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ibo_Latn.yaml' 2026-05-11T14:30:05,522 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-kin_Latn.yaml' 2026-05-11T14:30:05,523 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mey_Arab.yaml' 2026-05-11T14:30:05,524 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mlg_Latn.yaml' 2026-05-11T14:30:05,525 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-msa_Latn.yaml' 2026-05-11T14:30:05,526 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nde_Latn.yaml' 2026-05-11T14:30:05,528 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nso_Latn.yaml' 2026-05-11T14:30:05,529 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nya_Latn.yaml' 2026-05-11T14:30:05,530 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-orm_Ethi.yaml' 2026-05-11T14:30:05,531 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-shi_Arab.yaml' 2026-05-11T14:30:05,532 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-sna_Latn.yaml' 2026-05-11T14:30:05,534 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-som_Latn.yaml' 2026-05-11T14:30:05,535 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ssw_Latn.yaml' 2026-05-11T14:30:05,536 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-swa_Latn.yaml' 2026-05-11T14:30:05,538 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tam_Taml.yaml' 2026-05-11T14:30:05,539 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tel_Telu.yaml' 2026-05-11T14:30:05,541 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tir_Ethi.yaml' 2026-05-11T14:30:05,542 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ton_Latn.yaml' 2026-05-11T14:30:05,544 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tsn_Latn.yaml' 2026-05-11T14:30:05,545 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-urd_Arab.yaml' 2026-05-11T14:30:05,547 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ven_Latn.yaml' 2026-05-11T14:30:05,548 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-wol_Latn.yaml' 2026-05-11T14:30:05,549 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-xho_Latn.yaml' 2026-05-11T14:30:05,550 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-yor_Latn.yaml' 2026-05-11T14:30:05,552 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-zul_Latn.yaml' 2026-05-11T14:30:05,555 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex' 2026-05-11T14:30:05,556 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_afr_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,557 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_amh_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,559 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_arb_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,561 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_bem_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,562 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ewe_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,564 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_fra_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,565 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_hau_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,566 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ibo_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,567 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_kin_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,569 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mey_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,570 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mlg_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,572 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_msa_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,574 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nde_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,575 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nso_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,576 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nya_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,578 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_orm_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,580 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_shi_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,581 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_sna_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,582 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_som_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,583 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ssw_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,584 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_swa_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,586 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tam_Taml-eng_Latn.yaml' 2026-05-11T14:30:05,587 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tel_Telu-eng_Latn.yaml' 2026-05-11T14:30:05,588 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tir_Ethi-eng_Latn.yaml' 2026-05-11T14:30:05,589 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ton_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,591 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tsn_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,593 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_urd_Arab-eng_Latn.yaml' 2026-05-11T14:30:05,594 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ven_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,595 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_wol_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,596 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_xho_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,598 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_yor_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,599 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_zul_Latn-eng_Latn.yaml' 2026-05-11T14:30:05,601 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex' 2026-05-11T14:30:05,602 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-afr_Latn.yaml' 2026-05-11T14:30:05,603 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-amh_Ethi.yaml' 2026-05-11T14:30:05,605 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-arb_Arab.yaml' 2026-05-11T14:30:05,606 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-bem_Latn.yaml' 2026-05-11T14:30:05,607 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ewe_Latn.yaml' 2026-05-11T14:30:05,608 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-fra_Latn.yaml' 2026-05-11T14:30:05,610 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-hau_Latn.yaml' 2026-05-11T14:30:05,611 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ibo_Latn.yaml' 2026-05-11T14:30:05,612 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-kin_Latn.yaml' 2026-05-11T14:30:05,613 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mey_Arab.yaml' 2026-05-11T14:30:05,614 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mlg_Latn.yaml' 2026-05-11T14:30:05,616 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-msa_Latn.yaml' 2026-05-11T14:30:05,617 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nde_Latn.yaml' 2026-05-11T14:30:05,618 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nso_Latn.yaml' 2026-05-11T14:30:05,619 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nya_Latn.yaml' 2026-05-11T14:30:05,620 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-orm_Ethi.yaml' 2026-05-11T14:30:05,622 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-shi_Arab.yaml' 2026-05-11T14:30:05,623 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-sna_Latn.yaml' 2026-05-11T14:30:05,624 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-som_Latn.yaml' 2026-05-11T14:30:05,626 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ssw_Latn.yaml' 2026-05-11T14:30:05,627 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-swa_Latn.yaml' 2026-05-11T14:30:05,629 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tam_Taml.yaml' 2026-05-11T14:30:05,630 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tel_Telu.yaml' 2026-05-11T14:30:05,632 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tir_Ethi.yaml' 2026-05-11T14:30:05,633 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ton_Latn.yaml' 2026-05-11T14:30:05,634 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tsn_Latn.yaml' 2026-05-11T14:30:05,636 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-urd_Arab.yaml' 2026-05-11T14:30:05,637 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ven_Latn.yaml' 2026-05-11T14:30:05,638 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-wol_Latn.yaml' 2026-05-11T14:30:05,639 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-xho_Latn.yaml' 2026-05-11T14:30:05,640 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-yor_Latn.yaml' 2026-05-11T14:30:05,641 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-zul_Latn.yaml' 2026-05-11T14:30:05,644 adding 'lm_eval/tasks/afrobench/openai_mmlu/README.md' 2026-05-11T14:30:05,645 adding 'lm_eval/tasks/afrobench/openai_mmlu/openai_mmlu.yaml' 2026-05-11T14:30:05,646 adding 'lm_eval/tasks/afrobench/openai_mmlu/utils.py' 2026-05-11T14:30:05,648 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu' 2026-05-11T14:30:05,649 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_ara.yaml' 2026-05-11T14:30:05,650 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_swa.yaml' 2026-05-11T14:30:05,652 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_yor.yaml' 2026-05-11T14:30:05,653 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu' 2026-05-11T14:30:05,655 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_ara.yaml' 2026-05-11T14:30:05,656 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_swa.yaml' 2026-05-11T14:30:05,657 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_yor.yaml' 2026-05-11T14:30:05,659 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu' 2026-05-11T14:30:05,660 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_ara.yaml' 2026-05-11T14:30:05,661 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_swa.yaml' 2026-05-11T14:30:05,662 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_yor.yaml' 2026-05-11T14:30:05,664 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu' 2026-05-11T14:30:05,665 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_ara.yaml' 2026-05-11T14:30:05,666 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_swa.yaml' 2026-05-11T14:30:05,668 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_yor.yaml' 2026-05-11T14:30:05,669 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu' 2026-05-11T14:30:05,670 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_ara.yaml' 2026-05-11T14:30:05,671 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_swa.yaml' 2026-05-11T14:30:05,673 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_yor.yaml' 2026-05-11T14:30:05,674 adding 'lm_eval/tasks/afrobench/salt/README.md' 2026-05-11T14:30:05,676 adding 'lm_eval/tasks/afrobench/salt/gen_utils.py' 2026-05-11T14:30:05,677 adding 'lm_eval/tasks/afrobench/salt/salt.yaml' 2026-05-11T14:30:05,679 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt' 2026-05-11T14:30:05,680 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_ach-eng.yaml' 2026-05-11T14:30:05,681 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ach.yaml' 2026-05-11T14:30:05,682 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ibo.yaml' 2026-05-11T14:30:05,683 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lgg.yaml' 2026-05-11T14:30:05,685 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lug.yaml' 2026-05-11T14:30:05,686 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-nyn.yaml' 2026-05-11T14:30:05,687 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-swa.yaml' 2026-05-11T14:30:05,688 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-teo.yaml' 2026-05-11T14:30:05,689 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_ibo-eng.yaml' 2026-05-11T14:30:05,690 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_lgg-eng.yaml' 2026-05-11T14:30:05,691 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_lug-eng.yaml' 2026-05-11T14:30:05,693 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_nyn-eng.yaml' 2026-05-11T14:30:05,694 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_swa-eng.yaml' 2026-05-11T14:30:05,695 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_teo-eng.yaml' 2026-05-11T14:30:05,697 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt' 2026-05-11T14:30:05,698 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_ach-eng.yaml' 2026-05-11T14:30:05,700 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ach.yaml' 2026-05-11T14:30:05,701 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ibo.yaml' 2026-05-11T14:30:05,702 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lgg.yaml' 2026-05-11T14:30:05,703 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lug.yaml' 2026-05-11T14:30:05,704 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-nyn.yaml' 2026-05-11T14:30:05,706 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-swa.yaml' 2026-05-11T14:30:05,707 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-teo.yaml' 2026-05-11T14:30:05,708 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_ibo-eng.yaml' 2026-05-11T14:30:05,709 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_lgg-eng.yaml' 2026-05-11T14:30:05,711 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_lug-eng.yaml' 2026-05-11T14:30:05,712 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_nyn-eng.yaml' 2026-05-11T14:30:05,713 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_swa-eng.yaml' 2026-05-11T14:30:05,714 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_teo-eng.yaml' 2026-05-11T14:30:05,716 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt' 2026-05-11T14:30:05,717 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_ach-eng.yaml' 2026-05-11T14:30:05,718 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ach.yaml' 2026-05-11T14:30:05,720 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ibo.yaml' 2026-05-11T14:30:05,721 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lgg.yaml' 2026-05-11T14:30:05,722 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lug.yaml' 2026-05-11T14:30:05,723 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-nyn.yaml' 2026-05-11T14:30:05,724 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-swa.yaml' 2026-05-11T14:30:05,726 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-teo.yaml' 2026-05-11T14:30:05,727 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_ibo-eng.yaml' 2026-05-11T14:30:05,728 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_lgg-eng.yaml' 2026-05-11T14:30:05,729 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_lug-eng.yaml' 2026-05-11T14:30:05,730 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_nyn-eng.yaml' 2026-05-11T14:30:05,731 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_swa-eng.yaml' 2026-05-11T14:30:05,732 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_teo-eng.yaml' 2026-05-11T14:30:05,734 adding 'lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench.sh' 2026-05-11T14:30:05,735 adding 'lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench_lite.sh' 2026-05-11T14:30:05,737 adding 'lm_eval/tasks/afrobench/sib/README.md' 2026-05-11T14:30:05,738 adding 'lm_eval/tasks/afrobench/sib/sib.yaml' 2026-05-11T14:30:05,740 adding 'lm_eval/tasks/afrobench/sib/utils.py' 2026-05-11T14:30:05,742 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib' 2026-05-11T14:30:05,743 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_aeb.yaml' 2026-05-11T14:30:05,745 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_afr.yaml' 2026-05-11T14:30:05,746 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_aka.yaml' 2026-05-11T14:30:05,747 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_amh.yaml' 2026-05-11T14:30:05,748 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ary.yaml' 2026-05-11T14:30:05,749 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_arz.yaml' 2026-05-11T14:30:05,750 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_bam.yaml' 2026-05-11T14:30:05,752 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_bem.yaml' 2026-05-11T14:30:05,753 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_cjk.yaml' 2026-05-11T14:30:05,754 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_dik.yaml' 2026-05-11T14:30:05,755 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_dyu.yaml' 2026-05-11T14:30:05,756 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_eng.yaml' 2026-05-11T14:30:05,757 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ewe.yaml' 2026-05-11T14:30:05,759 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_fon.yaml' 2026-05-11T14:30:05,760 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_fra.yaml' 2026-05-11T14:30:05,761 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_fuv.yaml' 2026-05-11T14:30:05,762 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_gaz.yaml' 2026-05-11T14:30:05,763 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_hau.yaml' 2026-05-11T14:30:05,764 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ibo.yaml' 2026-05-11T14:30:05,766 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kab.yaml' 2026-05-11T14:30:05,767 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kam.yaml' 2026-05-11T14:30:05,768 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kbp.yaml' 2026-05-11T14:30:05,769 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kea.yaml' 2026-05-11T14:30:05,770 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kik.yaml' 2026-05-11T14:30:05,772 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kin.yaml' 2026-05-11T14:30:05,773 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kmb.yaml' 2026-05-11T14:30:05,774 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_knc.yaml' 2026-05-11T14:30:05,775 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kon.yaml' 2026-05-11T14:30:05,776 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_lin.yaml' 2026-05-11T14:30:05,778 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_lua.yaml' 2026-05-11T14:30:05,779 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_lug.yaml' 2026-05-11T14:30:05,780 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_luo.yaml' 2026-05-11T14:30:05,781 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_mos.yaml' 2026-05-11T14:30:05,782 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_nso.yaml' 2026-05-11T14:30:05,784 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_nus.yaml' 2026-05-11T14:30:05,785 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_nya.yaml' 2026-05-11T14:30:05,786 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_plt.yaml' 2026-05-11T14:30:05,787 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_por.yaml' 2026-05-11T14:30:05,788 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_run.yaml' 2026-05-11T14:30:05,789 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_sag.yaml' 2026-05-11T14:30:05,790 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_sna.yaml' 2026-05-11T14:30:05,791 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_som.yaml' 2026-05-11T14:30:05,792 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_sot.yaml' 2026-05-11T14:30:05,793 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ssw.yaml' 2026-05-11T14:30:05,794 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_swa.yaml' 2026-05-11T14:30:05,795 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_taq.yaml' 2026-05-11T14:30:05,797 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tir.yaml' 2026-05-11T14:30:05,798 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tso.yaml' 2026-05-11T14:30:05,799 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tum.yaml' 2026-05-11T14:30:05,800 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_twi.yaml' 2026-05-11T14:30:05,801 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tzm.yaml' 2026-05-11T14:30:05,802 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_umb.yaml' 2026-05-11T14:30:05,804 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_wol.yaml' 2026-05-11T14:30:05,805 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_xho.yaml' 2026-05-11T14:30:05,806 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_yor.yaml' 2026-05-11T14:30:05,807 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_zul.yaml' 2026-05-11T14:30:05,808 adding 'lm_eval/tasks/afrobench/sib/prompt_1/utils.py' 2026-05-11T14:30:05,811 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib' 2026-05-11T14:30:05,812 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_aeb.yaml' 2026-05-11T14:30:05,813 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_afr.yaml' 2026-05-11T14:30:05,815 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_aka.yaml' 2026-05-11T14:30:05,816 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_amh.yaml' 2026-05-11T14:30:05,818 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ary.yaml' 2026-05-11T14:30:05,819 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_arz.yaml' 2026-05-11T14:30:05,820 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_bam.yaml' 2026-05-11T14:30:05,821 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_bem.yaml' 2026-05-11T14:30:05,823 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_cjk.yaml' 2026-05-11T14:30:05,824 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_dik.yaml' 2026-05-11T14:30:05,825 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_dyu.yaml' 2026-05-11T14:30:05,826 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_eng.yaml' 2026-05-11T14:30:05,827 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ewe.yaml' 2026-05-11T14:30:05,829 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_fon.yaml' 2026-05-11T14:30:05,830 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_fra.yaml' 2026-05-11T14:30:05,831 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_fuv.yaml' 2026-05-11T14:30:05,832 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_gaz.yaml' 2026-05-11T14:30:05,833 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_hau.yaml' 2026-05-11T14:30:05,835 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ibo.yaml' 2026-05-11T14:30:05,836 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kab.yaml' 2026-05-11T14:30:05,837 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kam.yaml' 2026-05-11T14:30:05,838 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kbp.yaml' 2026-05-11T14:30:05,839 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kea.yaml' 2026-05-11T14:30:05,841 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kik.yaml' 2026-05-11T14:30:05,842 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kin.yaml' 2026-05-11T14:30:05,843 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kmb.yaml' 2026-05-11T14:30:05,844 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_knc.yaml' 2026-05-11T14:30:05,845 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kon.yaml' 2026-05-11T14:30:05,846 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_lin.yaml' 2026-05-11T14:30:05,847 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_lua.yaml' 2026-05-11T14:30:05,848 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_lug.yaml' 2026-05-11T14:30:05,850 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_luo.yaml' 2026-05-11T14:30:05,851 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_mos.yaml' 2026-05-11T14:30:05,852 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_nso.yaml' 2026-05-11T14:30:05,853 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_nus.yaml' 2026-05-11T14:30:05,854 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_nya.yaml' 2026-05-11T14:30:05,855 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_plt.yaml' 2026-05-11T14:30:05,856 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_por.yaml' 2026-05-11T14:30:05,857 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_run.yaml' 2026-05-11T14:30:05,858 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_sag.yaml' 2026-05-11T14:30:05,859 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_sna.yaml' 2026-05-11T14:30:05,861 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_som.yaml' 2026-05-11T14:30:05,862 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_sot.yaml' 2026-05-11T14:30:05,863 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ssw.yaml' 2026-05-11T14:30:05,864 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_swa.yaml' 2026-05-11T14:30:05,865 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_taq.yaml' 2026-05-11T14:30:05,866 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tir.yaml' 2026-05-11T14:30:05,867 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tso.yaml' 2026-05-11T14:30:05,869 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tum.yaml' 2026-05-11T14:30:05,870 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_twi.yaml' 2026-05-11T14:30:05,871 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tzm.yaml' 2026-05-11T14:30:05,872 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_umb.yaml' 2026-05-11T14:30:05,873 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_wol.yaml' 2026-05-11T14:30:05,875 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_xho.yaml' 2026-05-11T14:30:05,876 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_yor.yaml' 2026-05-11T14:30:05,877 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_zul.yaml' 2026-05-11T14:30:05,878 adding 'lm_eval/tasks/afrobench/sib/prompt_2/utils.py' 2026-05-11T14:30:05,881 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib' 2026-05-11T14:30:05,882 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_aeb.yaml' 2026-05-11T14:30:05,883 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_afr.yaml' 2026-05-11T14:30:05,885 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_aka.yaml' 2026-05-11T14:30:05,886 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_amh.yaml' 2026-05-11T14:30:05,887 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ary.yaml' 2026-05-11T14:30:05,889 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_arz.yaml' 2026-05-11T14:30:05,890 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_bam.yaml' 2026-05-11T14:30:05,891 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_bem.yaml' 2026-05-11T14:30:05,892 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_cjk.yaml' 2026-05-11T14:30:05,893 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_dik.yaml' 2026-05-11T14:30:05,895 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_dyu.yaml' 2026-05-11T14:30:05,896 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_eng.yaml' 2026-05-11T14:30:05,897 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ewe.yaml' 2026-05-11T14:30:05,899 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_fon.yaml' 2026-05-11T14:30:05,900 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_fra.yaml' 2026-05-11T14:30:05,901 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_fuv.yaml' 2026-05-11T14:30:05,902 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_gaz.yaml' 2026-05-11T14:30:05,903 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_hau.yaml' 2026-05-11T14:30:05,904 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ibo.yaml' 2026-05-11T14:30:05,905 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kab.yaml' 2026-05-11T14:30:05,906 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kam.yaml' 2026-05-11T14:30:05,908 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kbp.yaml' 2026-05-11T14:30:05,909 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kea.yaml' 2026-05-11T14:30:05,910 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kik.yaml' 2026-05-11T14:30:05,911 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kin.yaml' 2026-05-11T14:30:05,912 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kmb.yaml' 2026-05-11T14:30:05,914 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_knc.yaml' 2026-05-11T14:30:05,915 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kon.yaml' 2026-05-11T14:30:05,916 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_lin.yaml' 2026-05-11T14:30:05,917 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_lua.yaml' 2026-05-11T14:30:05,919 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_lug.yaml' 2026-05-11T14:30:05,920 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_luo.yaml' 2026-05-11T14:30:05,921 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_mos.yaml' 2026-05-11T14:30:05,922 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_nso.yaml' 2026-05-11T14:30:05,923 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_nus.yaml' 2026-05-11T14:30:05,924 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_nya.yaml' 2026-05-11T14:30:05,926 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_plt.yaml' 2026-05-11T14:30:05,927 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_por.yaml' 2026-05-11T14:30:05,928 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_run.yaml' 2026-05-11T14:30:05,929 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_sag.yaml' 2026-05-11T14:30:05,930 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_sna.yaml' 2026-05-11T14:30:05,932 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_som.yaml' 2026-05-11T14:30:05,933 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_sot.yaml' 2026-05-11T14:30:05,934 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ssw.yaml' 2026-05-11T14:30:05,935 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_swa.yaml' 2026-05-11T14:30:05,936 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_taq.yaml' 2026-05-11T14:30:05,938 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tir.yaml' 2026-05-11T14:30:05,939 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tso.yaml' 2026-05-11T14:30:05,940 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tum.yaml' 2026-05-11T14:30:05,941 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_twi.yaml' 2026-05-11T14:30:05,942 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tzm.yaml' 2026-05-11T14:30:05,944 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_umb.yaml' 2026-05-11T14:30:05,945 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_wol.yaml' 2026-05-11T14:30:05,946 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_xho.yaml' 2026-05-11T14:30:05,947 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_yor.yaml' 2026-05-11T14:30:05,949 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_zul.yaml' 2026-05-11T14:30:05,950 adding 'lm_eval/tasks/afrobench/sib/prompt_3/utils.py' 2026-05-11T14:30:05,952 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib' 2026-05-11T14:30:05,954 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_aeb.yaml' 2026-05-11T14:30:05,955 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_afr.yaml' 2026-05-11T14:30:05,956 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_aka.yaml' 2026-05-11T14:30:05,957 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_amh.yaml' 2026-05-11T14:30:05,959 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ary.yaml' 2026-05-11T14:30:05,960 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_arz.yaml' 2026-05-11T14:30:05,961 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_bam.yaml' 2026-05-11T14:30:05,962 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_bem.yaml' 2026-05-11T14:30:05,963 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_cjk.yaml' 2026-05-11T14:30:05,964 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_dik.yaml' 2026-05-11T14:30:05,965 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_dyu.yaml' 2026-05-11T14:30:05,966 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_eng.yaml' 2026-05-11T14:30:05,968 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ewe.yaml' 2026-05-11T14:30:05,969 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_fon.yaml' 2026-05-11T14:30:05,970 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_fra.yaml' 2026-05-11T14:30:05,971 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_fuv.yaml' 2026-05-11T14:30:05,972 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_gaz.yaml' 2026-05-11T14:30:05,973 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_hau.yaml' 2026-05-11T14:30:05,974 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ibo.yaml' 2026-05-11T14:30:05,975 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kab.yaml' 2026-05-11T14:30:05,977 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kam.yaml' 2026-05-11T14:30:05,978 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kbp.yaml' 2026-05-11T14:30:05,979 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kea.yaml' 2026-05-11T14:30:05,980 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kik.yaml' 2026-05-11T14:30:05,981 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kin.yaml' 2026-05-11T14:30:05,982 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kmb.yaml' 2026-05-11T14:30:05,983 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_knc.yaml' 2026-05-11T14:30:05,985 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kon.yaml' 2026-05-11T14:30:05,986 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_lin.yaml' 2026-05-11T14:30:05,987 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_lua.yaml' 2026-05-11T14:30:05,988 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_lug.yaml' 2026-05-11T14:30:05,989 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_luo.yaml' 2026-05-11T14:30:05,990 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_mos.yaml' 2026-05-11T14:30:05,992 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_nso.yaml' 2026-05-11T14:30:05,993 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_nus.yaml' 2026-05-11T14:30:05,994 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_nya.yaml' 2026-05-11T14:30:05,995 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_plt.yaml' 2026-05-11T14:30:05,996 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_por.yaml' 2026-05-11T14:30:05,998 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_run.yaml' 2026-05-11T14:30:05,999 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_sag.yaml' 2026-05-11T14:30:06,000 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_sna.yaml' 2026-05-11T14:30:06,001 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_som.yaml' 2026-05-11T14:30:06,002 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_sot.yaml' 2026-05-11T14:30:06,004 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ssw.yaml' 2026-05-11T14:30:06,005 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_swa.yaml' 2026-05-11T14:30:06,006 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_taq.yaml' 2026-05-11T14:30:06,007 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tir.yaml' 2026-05-11T14:30:06,008 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tso.yaml' 2026-05-11T14:30:06,010 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tum.yaml' 2026-05-11T14:30:06,011 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_twi.yaml' 2026-05-11T14:30:06,012 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tzm.yaml' 2026-05-11T14:30:06,013 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_umb.yaml' 2026-05-11T14:30:06,015 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_wol.yaml' 2026-05-11T14:30:06,016 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_xho.yaml' 2026-05-11T14:30:06,017 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_yor.yaml' 2026-05-11T14:30:06,018 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_zul.yaml' 2026-05-11T14:30:06,019 adding 'lm_eval/tasks/afrobench/sib/prompt_4/utils.py' 2026-05-11T14:30:06,022 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib' 2026-05-11T14:30:06,023 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_aeb.yaml' 2026-05-11T14:30:06,024 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_afr.yaml' 2026-05-11T14:30:06,025 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_aka.yaml' 2026-05-11T14:30:06,026 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_amh.yaml' 2026-05-11T14:30:06,028 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ary.yaml' 2026-05-11T14:30:06,029 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_arz.yaml' 2026-05-11T14:30:06,030 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_bam.yaml' 2026-05-11T14:30:06,031 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_bem.yaml' 2026-05-11T14:30:06,032 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_cjk.yaml' 2026-05-11T14:30:06,034 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_dik.yaml' 2026-05-11T14:30:06,035 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_dyu.yaml' 2026-05-11T14:30:06,036 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_eng.yaml' 2026-05-11T14:30:06,037 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ewe.yaml' 2026-05-11T14:30:06,039 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_fon.yaml' 2026-05-11T14:30:06,040 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_fra.yaml' 2026-05-11T14:30:06,041 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_fuv.yaml' 2026-05-11T14:30:06,042 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_gaz.yaml' 2026-05-11T14:30:06,044 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_hau.yaml' 2026-05-11T14:30:06,045 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ibo.yaml' 2026-05-11T14:30:06,046 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kab.yaml' 2026-05-11T14:30:06,047 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kam.yaml' 2026-05-11T14:30:06,049 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kbp.yaml' 2026-05-11T14:30:06,050 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kea.yaml' 2026-05-11T14:30:06,051 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kik.yaml' 2026-05-11T14:30:06,052 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kin.yaml' 2026-05-11T14:30:06,054 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kmb.yaml' 2026-05-11T14:30:06,055 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_knc.yaml' 2026-05-11T14:30:06,056 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kon.yaml' 2026-05-11T14:30:06,058 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_lin.yaml' 2026-05-11T14:30:06,059 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_lua.yaml' 2026-05-11T14:30:06,060 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_lug.yaml' 2026-05-11T14:30:06,061 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_luo.yaml' 2026-05-11T14:30:06,063 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_mos.yaml' 2026-05-11T14:30:06,064 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_nso.yaml' 2026-05-11T14:30:06,066 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_nus.yaml' 2026-05-11T14:30:06,067 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_nya.yaml' 2026-05-11T14:30:06,068 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_plt.yaml' 2026-05-11T14:30:06,070 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_por.yaml' 2026-05-11T14:30:06,071 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_run.yaml' 2026-05-11T14:30:06,072 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_sag.yaml' 2026-05-11T14:30:06,073 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_sna.yaml' 2026-05-11T14:30:06,075 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_som.yaml' 2026-05-11T14:30:06,076 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_sot.yaml' 2026-05-11T14:30:06,077 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ssw.yaml' 2026-05-11T14:30:06,079 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_swa.yaml' 2026-05-11T14:30:06,080 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_taq.yaml' 2026-05-11T14:30:06,081 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tir.yaml' 2026-05-11T14:30:06,082 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tso.yaml' 2026-05-11T14:30:06,083 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tum.yaml' 2026-05-11T14:30:06,084 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_twi.yaml' 2026-05-11T14:30:06,086 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tzm.yaml' 2026-05-11T14:30:06,087 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_umb.yaml' 2026-05-11T14:30:06,088 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_wol.yaml' 2026-05-11T14:30:06,089 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_xho.yaml' 2026-05-11T14:30:06,090 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_yor.yaml' 2026-05-11T14:30:06,092 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_zul.yaml' 2026-05-11T14:30:06,093 adding 'lm_eval/tasks/afrobench/sib/prompt_5/utils.py' 2026-05-11T14:30:06,095 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/README.md' 2026-05-11T14:30:06,096 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/uhura.yaml' 2026-05-11T14:30:06,097 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/utils.py' 2026-05-11T14:30:06,099 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy' 2026-05-11T14:30:06,101 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_am.yaml' 2026-05-11T14:30:06,102 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_en.yaml' 2026-05-11T14:30:06,103 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_ha.yaml' 2026-05-11T14:30:06,104 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_nso.yaml' 2026-05-11T14:30:06,105 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_sw.yaml' 2026-05-11T14:30:06,107 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_yo.yaml' 2026-05-11T14:30:06,108 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_zu.yaml' 2026-05-11T14:30:06,109 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/utils.py' 2026-05-11T14:30:06,110 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy' 2026-05-11T14:30:06,112 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_am.yaml' 2026-05-11T14:30:06,113 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_en.yaml' 2026-05-11T14:30:06,114 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_ha.yaml' 2026-05-11T14:30:06,115 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_nso.yaml' 2026-05-11T14:30:06,116 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_sw.yaml' 2026-05-11T14:30:06,118 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_yo.yaml' 2026-05-11T14:30:06,119 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_zu.yaml' 2026-05-11T14:30:06,120 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/utils.py' 2026-05-11T14:30:06,122 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy' 2026-05-11T14:30:06,123 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_am.yaml' 2026-05-11T14:30:06,124 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_en.yaml' 2026-05-11T14:30:06,125 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_ha.yaml' 2026-05-11T14:30:06,127 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_nso.yaml' 2026-05-11T14:30:06,128 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_sw.yaml' 2026-05-11T14:30:06,129 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_yo.yaml' 2026-05-11T14:30:06,130 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_zu.yaml' 2026-05-11T14:30:06,131 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/utils.py' 2026-05-11T14:30:06,133 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy' 2026-05-11T14:30:06,134 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_am.yaml' 2026-05-11T14:30:06,136 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_en.yaml' 2026-05-11T14:30:06,137 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_ha.yaml' 2026-05-11T14:30:06,138 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_nso.yaml' 2026-05-11T14:30:06,139 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_sw.yaml' 2026-05-11T14:30:06,140 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_yo.yaml' 2026-05-11T14:30:06,141 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_zu.yaml' 2026-05-11T14:30:06,142 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/utils.py' 2026-05-11T14:30:06,144 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy' 2026-05-11T14:30:06,145 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_am.yaml' 2026-05-11T14:30:06,147 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_en.yaml' 2026-05-11T14:30:06,148 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_ha.yaml' 2026-05-11T14:30:06,149 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_nso.yaml' 2026-05-11T14:30:06,150 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_sw.yaml' 2026-05-11T14:30:06,151 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_yo.yaml' 2026-05-11T14:30:06,152 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_zu.yaml' 2026-05-11T14:30:06,154 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/utils.py' 2026-05-11T14:30:06,155 adding 'lm_eval/tasks/afrobench/xlsum/README.md' 2026-05-11T14:30:06,157 adding 'lm_eval/tasks/afrobench/xlsum/utils.py' 2026-05-11T14:30:06,158 adding 'lm_eval/tasks/afrobench/xlsum/xlsum.yaml' 2026-05-11T14:30:06,160 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/utils.py' 2026-05-11T14:30:06,161 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum' 2026-05-11T14:30:06,162 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_amharic.yaml' 2026-05-11T14:30:06,163 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_arabic.yaml' 2026-05-11T14:30:06,164 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_hausa.yaml' 2026-05-11T14:30:06,166 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_igbo.yaml' 2026-05-11T14:30:06,167 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_kirundi.yaml' 2026-05-11T14:30:06,168 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_oromo.yaml' 2026-05-11T14:30:06,169 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_pidgin.yaml' 2026-05-11T14:30:06,170 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_somali.yaml' 2026-05-11T14:30:06,172 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_swahili.yaml' 2026-05-11T14:30:06,173 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_telugu.yaml' 2026-05-11T14:30:06,174 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_tigrinya.yaml' 2026-05-11T14:30:06,175 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_yoruba.yaml' 2026-05-11T14:30:06,177 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/utils.py' 2026-05-11T14:30:06,178 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum' 2026-05-11T14:30:06,179 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_amharic.yaml' 2026-05-11T14:30:06,181 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_arabic.yaml' 2026-05-11T14:30:06,182 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_hausa.yaml' 2026-05-11T14:30:06,183 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_igbo.yaml' 2026-05-11T14:30:06,184 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_kirundi.yaml' 2026-05-11T14:30:06,185 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_oromo.yaml' 2026-05-11T14:30:06,187 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_pidgin.yaml' 2026-05-11T14:30:06,188 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_somali.yaml' 2026-05-11T14:30:06,189 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_swahili.yaml' 2026-05-11T14:30:06,190 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_telugu.yaml' 2026-05-11T14:30:06,191 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_tigrinya.yaml' 2026-05-11T14:30:06,192 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_yoruba.yaml' 2026-05-11T14:30:06,194 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/utils.py' 2026-05-11T14:30:06,195 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum' 2026-05-11T14:30:06,196 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_amharic.yaml' 2026-05-11T14:30:06,197 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_arabic.yaml' 2026-05-11T14:30:06,198 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_hausa.yaml' 2026-05-11T14:30:06,199 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_igbo.yaml' 2026-05-11T14:30:06,201 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_kirundi.yaml' 2026-05-11T14:30:06,202 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_oromo.yaml' 2026-05-11T14:30:06,203 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_pidgin.yaml' 2026-05-11T14:30:06,204 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_somali.yaml' 2026-05-11T14:30:06,205 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_swahili.yaml' 2026-05-11T14:30:06,206 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_telugu.yaml' 2026-05-11T14:30:06,208 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_tigrinya.yaml' 2026-05-11T14:30:06,209 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_yoruba.yaml' 2026-05-11T14:30:06,211 adding 'lm_eval/tasks/agieval/README.md' 2026-05-11T14:30:06,212 adding 'lm_eval/tasks/agieval/agieval.yaml' 2026-05-11T14:30:06,214 adding 'lm_eval/tasks/agieval/agieval_cn.yaml' 2026-05-11T14:30:06,215 adding 'lm_eval/tasks/agieval/agieval_en.yaml' 2026-05-11T14:30:06,216 adding 'lm_eval/tasks/agieval/agieval_nous.yaml' 2026-05-11T14:30:06,217 adding 'lm_eval/tasks/agieval/aqua-rat.yaml' 2026-05-11T14:30:06,219 adding 'lm_eval/tasks/agieval/gaokao-biology.yaml' 2026-05-11T14:30:06,220 adding 'lm_eval/tasks/agieval/gaokao-chemistry.yaml' 2026-05-11T14:30:06,221 adding 'lm_eval/tasks/agieval/gaokao-chinese.yaml' 2026-05-11T14:30:06,222 adding 'lm_eval/tasks/agieval/gaokao-english.yaml' 2026-05-11T14:30:06,223 adding 'lm_eval/tasks/agieval/gaokao-geography.yaml' 2026-05-11T14:30:06,224 adding 'lm_eval/tasks/agieval/gaokao-history.yaml' 2026-05-11T14:30:06,225 adding 'lm_eval/tasks/agieval/gaokao-mathcloze.yaml' 2026-05-11T14:30:06,227 adding 'lm_eval/tasks/agieval/gaokao-mathqa.yaml' 2026-05-11T14:30:06,228 adding 'lm_eval/tasks/agieval/gaokao-physics.yaml' 2026-05-11T14:30:06,229 adding 'lm_eval/tasks/agieval/jec-qa-ca.yaml' 2026-05-11T14:30:06,230 adding 'lm_eval/tasks/agieval/jec-qa-kd.yaml' 2026-05-11T14:30:06,231 adding 'lm_eval/tasks/agieval/logiqa-en.yaml' 2026-05-11T14:30:06,232 adding 'lm_eval/tasks/agieval/logiqa-zh.yaml' 2026-05-11T14:30:06,233 adding 'lm_eval/tasks/agieval/lsat-ar.yaml' 2026-05-11T14:30:06,235 adding 'lm_eval/tasks/agieval/lsat-lr.yaml' 2026-05-11T14:30:06,236 adding 'lm_eval/tasks/agieval/lsat-rc.yaml' 2026-05-11T14:30:06,237 adding 'lm_eval/tasks/agieval/math.yaml' 2026-05-11T14:30:06,238 adding 'lm_eval/tasks/agieval/sat-en-without-passage.yaml' 2026-05-11T14:30:06,239 adding 'lm_eval/tasks/agieval/sat-en.yaml' 2026-05-11T14:30:06,240 adding 'lm_eval/tasks/agieval/sat-math.yaml' 2026-05-11T14:30:06,242 adding 'lm_eval/tasks/agieval/utils.py' 2026-05-11T14:30:06,244 adding 'lm_eval/tasks/aime/README.md' 2026-05-11T14:30:06,245 adding 'lm_eval/tasks/aime/aime.yaml' 2026-05-11T14:30:06,247 adding 'lm_eval/tasks/aime/aime24.yaml' 2026-05-11T14:30:06,248 adding 'lm_eval/tasks/aime/aime25.yaml' 2026-05-11T14:30:06,249 adding 'lm_eval/tasks/aime/utils.py' 2026-05-11T14:30:06,252 adding 'lm_eval/tasks/alghafa/copa_ar/README.md' 2026-05-11T14:30:06,253 adding 'lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml' 2026-05-11T14:30:06,254 adding 'lm_eval/tasks/alghafa/piqa_ar/README.md' 2026-05-11T14:30:06,256 adding 'lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml' 2026-05-11T14:30:06,257 adding 'lm_eval/tasks/anli/README.md' 2026-05-11T14:30:06,259 adding 'lm_eval/tasks/anli/anli_r1.yaml' 2026-05-11T14:30:06,260 adding 'lm_eval/tasks/anli/anli_r2.yaml' 2026-05-11T14:30:06,261 adding 'lm_eval/tasks/anli/anli_r3.yaml' 2026-05-11T14:30:06,263 adding 'lm_eval/tasks/arab_culture/README.md' 2026-05-11T14:30:06,264 adding 'lm_eval/tasks/arab_culture/_arab_culture.yaml' 2026-05-11T14:30:06,265 adding 'lm_eval/tasks/arab_culture/_arab_culture_gulf.yaml' 2026-05-11T14:30:06,266 adding 'lm_eval/tasks/arab_culture/_arab_culture_levant.yaml' 2026-05-11T14:30:06,267 adding 'lm_eval/tasks/arab_culture/_arab_culture_nile_valley.yaml' 2026-05-11T14:30:06,269 adding 'lm_eval/tasks/arab_culture/_arab_culture_north_africa.yaml' 2026-05-11T14:30:06,270 adding 'lm_eval/tasks/arab_culture/_default_arab_culture_mcq_template_yaml' 2026-05-11T14:30:06,271 adding 'lm_eval/tasks/arab_culture/_generate_configs.py' 2026-05-11T14:30:06,272 adding 'lm_eval/tasks/arab_culture/arab_culture_algeria.yaml' 2026-05-11T14:30:06,273 adding 'lm_eval/tasks/arab_culture/arab_culture_egypt.yaml' 2026-05-11T14:30:06,274 adding 'lm_eval/tasks/arab_culture/arab_culture_jordan.yaml' 2026-05-11T14:30:06,275 adding 'lm_eval/tasks/arab_culture/arab_culture_ksa.yaml' 2026-05-11T14:30:06,277 adding 'lm_eval/tasks/arab_culture/arab_culture_lebanon.yaml' 2026-05-11T14:30:06,278 adding 'lm_eval/tasks/arab_culture/arab_culture_libya.yaml' 2026-05-11T14:30:06,279 adding 'lm_eval/tasks/arab_culture/arab_culture_morocco.yaml' 2026-05-11T14:30:06,280 adding 'lm_eval/tasks/arab_culture/arab_culture_palestine.yaml' 2026-05-11T14:30:06,281 adding 'lm_eval/tasks/arab_culture/arab_culture_sudan.yaml' 2026-05-11T14:30:06,282 adding 'lm_eval/tasks/arab_culture/arab_culture_syria.yaml' 2026-05-11T14:30:06,283 adding 'lm_eval/tasks/arab_culture/arab_culture_tunisia.yaml' 2026-05-11T14:30:06,285 adding 'lm_eval/tasks/arab_culture/arab_culture_uae.yaml' 2026-05-11T14:30:06,286 adding 'lm_eval/tasks/arab_culture/arab_culture_yemen.yaml' 2026-05-11T14:30:06,288 adding 'lm_eval/tasks/arab_culture/prompts.py' 2026-05-11T14:30:06,289 adding 'lm_eval/tasks/arab_culture/utils_mcq.py' 2026-05-11T14:30:06,291 adding 'lm_eval/tasks/arab_culture_completion/README.md' 2026-05-11T14:30:06,293 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion.yaml' 2026-05-11T14:30:06,294 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_gulf.yaml' 2026-05-11T14:30:06,295 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_levant.yaml' 2026-05-11T14:30:06,296 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_nile_valley.yaml' 2026-05-11T14:30:06,297 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_north_africa.yaml' 2026-05-11T14:30:06,299 adding 'lm_eval/tasks/arab_culture_completion/_default_arab_culture_completion_template_yaml' 2026-05-11T14:30:06,300 adding 'lm_eval/tasks/arab_culture_completion/_generate_configs.py' 2026-05-11T14:30:06,301 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_algeria.yaml' 2026-05-11T14:30:06,302 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_egypt.yaml' 2026-05-11T14:30:06,303 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_jordan.yaml' 2026-05-11T14:30:06,305 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_ksa.yaml' 2026-05-11T14:30:06,306 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_lebanon.yaml' 2026-05-11T14:30:06,307 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_libya.yaml' 2026-05-11T14:30:06,308 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_morocco.yaml' 2026-05-11T14:30:06,309 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_palestine.yaml' 2026-05-11T14:30:06,311 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_sudan.yaml' 2026-05-11T14:30:06,312 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_syria.yaml' 2026-05-11T14:30:06,313 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_tunisia.yaml' 2026-05-11T14:30:06,314 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_uae.yaml' 2026-05-11T14:30:06,315 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_yemen.yaml' 2026-05-11T14:30:06,316 adding 'lm_eval/tasks/arab_culture_completion/prompts.py' 2026-05-11T14:30:06,318 adding 'lm_eval/tasks/arab_culture_completion/utils_completion.py' 2026-05-11T14:30:06,321 adding 'lm_eval/tasks/arabic_leaderboard_complete/README.md' 2026-05-11T14:30:06,322 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_complete.yaml' 2026-05-11T14:30:06,323 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa.yaml' 2026-05-11T14:30:06,325 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_mcq_exams_test_ar.yaml' 2026-05-11T14:30:06,326 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_dialects.yaml' 2026-05-11T14:30:06,327 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_msa.yaml' 2026-05-11T14:30:06,328 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task.yaml' 2026-05-11T14:30:06,329 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task.yaml' 2026-05-11T14:30:06,331 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task.yaml' 2026-05-11T14:30:06,332 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task.yaml' 2026-05-11T14:30:06,333 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task.yaml' 2026-05-11T14:30:06,334 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_sentiment_task.yaml' 2026-05-11T14:30:06,335 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/utils.py' 2026-05-11T14:30:06,337 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_exams.yaml' 2026-05-11T14:30:06,338 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_leaderboard_arabic_exams.yaml' 2026-05-11T14:30:06,340 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/utils.py' 2026-05-11T14:30:06,343 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu.yaml' 2026-05-11T14:30:06,344 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_abstract_algebra.yaml' 2026-05-11T14:30:06,345 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_anatomy.yaml' 2026-05-11T14:30:06,346 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_astronomy.yaml' 2026-05-11T14:30:06,348 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_business_ethics.yaml' 2026-05-11T14:30:06,349 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:06,350 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_biology.yaml' 2026-05-11T14:30:06,351 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_chemistry.yaml' 2026-05-11T14:30:06,352 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_computer_science.yaml' 2026-05-11T14:30:06,353 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_mathematics.yaml' 2026-05-11T14:30:06,354 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_medicine.yaml' 2026-05-11T14:30:06,356 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_physics.yaml' 2026-05-11T14:30:06,357 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_computer_security.yaml' 2026-05-11T14:30:06,358 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_conceptual_physics.yaml' 2026-05-11T14:30:06,359 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_econometrics.yaml' 2026-05-11T14:30:06,360 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_electrical_engineering.yaml' 2026-05-11T14:30:06,362 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:06,363 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_formal_logic.yaml' 2026-05-11T14:30:06,364 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_global_facts.yaml' 2026-05-11T14:30:06,365 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_biology.yaml' 2026-05-11T14:30:06,366 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:06,368 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:06,369 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_european_history.yaml' 2026-05-11T14:30:06,370 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_geography.yaml' 2026-05-11T14:30:06,371 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:06,372 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:06,374 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:06,375 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:06,376 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_physics.yaml' 2026-05-11T14:30:06,377 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_psychology.yaml' 2026-05-11T14:30:06,379 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_statistics.yaml' 2026-05-11T14:30:06,380 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_us_history.yaml' 2026-05-11T14:30:06,381 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_world_history.yaml' 2026-05-11T14:30:06,382 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_aging.yaml' 2026-05-11T14:30:06,383 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_sexuality.yaml' 2026-05-11T14:30:06,384 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_international_law.yaml' 2026-05-11T14:30:06,385 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_jurisprudence.yaml' 2026-05-11T14:30:06,387 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_logical_fallacies.yaml' 2026-05-11T14:30:06,388 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_machine_learning.yaml' 2026-05-11T14:30:06,389 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_management.yaml' 2026-05-11T14:30:06,390 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_marketing.yaml' 2026-05-11T14:30:06,391 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_medical_genetics.yaml' 2026-05-11T14:30:06,393 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_miscellaneous.yaml' 2026-05-11T14:30:06,394 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_disputes.yaml' 2026-05-11T14:30:06,395 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_scenarios.yaml' 2026-05-11T14:30:06,396 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_nutrition.yaml' 2026-05-11T14:30:06,397 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_philosophy.yaml' 2026-05-11T14:30:06,399 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_prehistory.yaml' 2026-05-11T14:30:06,400 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_accounting.yaml' 2026-05-11T14:30:06,401 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_law.yaml' 2026-05-11T14:30:06,402 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_medicine.yaml' 2026-05-11T14:30:06,403 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_psychology.yaml' 2026-05-11T14:30:06,405 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_public_relations.yaml' 2026-05-11T14:30:06,406 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_security_studies.yaml' 2026-05-11T14:30:06,407 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_sociology.yaml' 2026-05-11T14:30:06,408 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:06,409 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_virology.yaml' 2026-05-11T14:30:06,411 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_world_religions.yaml' 2026-05-11T14:30:06,412 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/utils.py' 2026-05-11T14:30:06,413 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_leaderboard_arabic_mt_arc_challenge.yaml' 2026-05-11T14:30:06,414 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_mt_arc_challenge.yaml' 2026-05-11T14:30:06,416 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/utils.py' 2026-05-11T14:30:06,417 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_leaderboard_arabic_mt_arc_easy.yaml' 2026-05-11T14:30:06,418 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_mt_arc_easy.yaml' 2026-05-11T14:30:06,420 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/utils.py' 2026-05-11T14:30:06,421 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_leaderboard_arabic_mt_boolq.yaml' 2026-05-11T14:30:06,422 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_mt_boolq.yaml' 2026-05-11T14:30:06,424 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/utils.py' 2026-05-11T14:30:06,425 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_leaderboard_arabic_mt_copa.yaml' 2026-05-11T14:30:06,427 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_mt_copa.yaml' 2026-05-11T14:30:06,428 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/utils.py' 2026-05-11T14:30:06,430 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_leaderboard_arabic_mt_hellaswag.yaml' 2026-05-11T14:30:06,431 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_mt_hellaswag.yaml' 2026-05-11T14:30:06,432 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/utils.py' 2026-05-11T14:30:06,434 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_leaderboard_arabic_mt_mmlu.yaml' 2026-05-11T14:30:06,435 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_mt_mmlu.yaml' 2026-05-11T14:30:06,436 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/utils.py' 2026-05-11T14:30:06,438 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_leaderboard_arabic_mt_openbook_qa.yaml' 2026-05-11T14:30:06,439 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_mt_openbook_qa.yaml' 2026-05-11T14:30:06,440 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/utils.py' 2026-05-11T14:30:06,442 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_leaderboard_arabic_mt_piqa.yaml' 2026-05-11T14:30:06,443 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_mt_piqa.yaml' 2026-05-11T14:30:06,445 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/utils.py' 2026-05-11T14:30:06,446 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_leaderboard_arabic_mt_race.yaml' 2026-05-11T14:30:06,448 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_mt_race.yaml' 2026-05-11T14:30:06,449 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/utils.py' 2026-05-11T14:30:06,451 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_leaderboard_arabic_mt_sciq.yaml' 2026-05-11T14:30:06,452 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_mt_sciq.yaml' 2026-05-11T14:30:06,453 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/utils.py' 2026-05-11T14:30:06,454 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_leaderboard_arabic_mt_toxigen.yaml' 2026-05-11T14:30:06,456 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_mt_toxigen.yaml' 2026-05-11T14:30:06,457 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/utils.py' 2026-05-11T14:30:06,459 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva.yaml' 2026-05-11T14:30:06,460 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Algeria.yaml' 2026-05-11T14:30:06,462 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Ancient_Egypt.yaml' 2026-05-11T14:30:06,463 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arab_Empire.yaml' 2026-05-11T14:30:06,464 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Architecture.yaml' 2026-05-11T14:30:06,465 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Art.yaml' 2026-05-11T14:30:06,466 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Astronomy.yaml' 2026-05-11T14:30:06,468 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Calligraphy.yaml' 2026-05-11T14:30:06,469 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ceremony.yaml' 2026-05-11T14:30:06,470 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Clothing.yaml' 2026-05-11T14:30:06,471 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Culture.yaml' 2026-05-11T14:30:06,472 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Food.yaml' 2026-05-11T14:30:06,474 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Funeral.yaml' 2026-05-11T14:30:06,475 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Geography.yaml' 2026-05-11T14:30:06,476 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_History.yaml' 2026-05-11T14:30:06,477 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Language_Origin.yaml' 2026-05-11T14:30:06,479 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Literature.yaml' 2026-05-11T14:30:06,480 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Math.yaml' 2026-05-11T14:30:06,481 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Medicine.yaml' 2026-05-11T14:30:06,482 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Music.yaml' 2026-05-11T14:30:06,484 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ornament.yaml' 2026-05-11T14:30:06,485 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Philosophy.yaml' 2026-05-11T14:30:06,486 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry.yaml' 2026-05-11T14:30:06,487 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Wedding.yaml' 2026-05-11T14:30:06,489 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Bahrain.yaml' 2026-05-11T14:30:06,490 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Comoros.yaml' 2026-05-11T14:30:06,491 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Egypt_modern.yaml' 2026-05-11T14:30:06,492 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromAncientEgypt.yaml' 2026-05-11T14:30:06,493 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromByzantium.yaml' 2026-05-11T14:30:06,495 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromChina.yaml' 2026-05-11T14:30:06,496 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromGreece.yaml' 2026-05-11T14:30:06,497 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromIslam.yaml' 2026-05-11T14:30:06,498 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromPersia.yaml' 2026-05-11T14:30:06,499 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromRome.yaml' 2026-05-11T14:30:06,501 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Iraq.yaml' 2026-05-11T14:30:06,502 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_Education.yaml' 2026-05-11T14:30:06,503 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_branches_and_schools.yaml' 2026-05-11T14:30:06,504 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islamic_law_system.yaml' 2026-05-11T14:30:06,505 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Jordan.yaml' 2026-05-11T14:30:06,506 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Kuwait.yaml' 2026-05-11T14:30:06,508 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Lebanon.yaml' 2026-05-11T14:30:06,509 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Libya.yaml' 2026-05-11T14:30:06,510 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mauritania.yaml' 2026-05-11T14:30:06,511 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mesopotamia_civilization.yaml' 2026-05-11T14:30:06,512 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Morocco.yaml' 2026-05-11T14:30:06,514 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Oman.yaml' 2026-05-11T14:30:06,515 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Palestine.yaml' 2026-05-11T14:30:06,516 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Qatar.yaml' 2026-05-11T14:30:06,517 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Saudi_Arabia.yaml' 2026-05-11T14:30:06,518 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Somalia.yaml' 2026-05-11T14:30:06,520 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Sudan.yaml' 2026-05-11T14:30:06,521 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Syria.yaml' 2026-05-11T14:30:06,522 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Tunisia.yaml' 2026-05-11T14:30:06,523 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_United_Arab_Emirates.yaml' 2026-05-11T14:30:06,524 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Yemen.yaml' 2026-05-11T14:30:06,525 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_communication.yaml' 2026-05-11T14:30:06,526 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_computer_and_phone.yaml' 2026-05-11T14:30:06,528 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_daily_life.yaml' 2026-05-11T14:30:06,529 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_entertainment.yaml' 2026-05-11T14:30:06,530 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/utils.py' 2026-05-11T14:30:06,532 adding 'lm_eval/tasks/arabic_leaderboard_light/README.md' 2026-05-11T14:30:06,533 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_light.yaml' 2026-05-11T14:30:06,535 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_light.yaml' 2026-05-11T14:30:06,536 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_mcq_exams_test_ar_light.yaml' 2026-05-11T14:30:06,537 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_dialects_light.yaml' 2026-05-11T14:30:06,538 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_msa_light.yaml' 2026-05-11T14:30:06,540 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task_light.yaml' 2026-05-11T14:30:06,541 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task_light.yaml' 2026-05-11T14:30:06,542 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task_light.yaml' 2026-05-11T14:30:06,543 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task_light.yaml' 2026-05-11T14:30:06,545 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task_light.yaml' 2026-05-11T14:30:06,546 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_sentiment_task_light.yaml' 2026-05-11T14:30:06,547 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/utils.py' 2026-05-11T14:30:06,549 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_exams_light.yaml' 2026-05-11T14:30:06,550 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_leaderboard_arabic_exams_light.yaml' 2026-05-11T14:30:06,551 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/utils.py' 2026-05-11T14:30:06,554 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_abstract_algebra_light.yaml' 2026-05-11T14:30:06,555 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_anatomy_light.yaml' 2026-05-11T14:30:06,557 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_astronomy_light.yaml' 2026-05-11T14:30:06,558 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_business_ethics_light.yaml' 2026-05-11T14:30:06,559 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_clinical_knowledge_light.yaml' 2026-05-11T14:30:06,560 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_biology_light.yaml' 2026-05-11T14:30:06,561 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_chemistry_light.yaml' 2026-05-11T14:30:06,562 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_computer_science_light.yaml' 2026-05-11T14:30:06,564 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_mathematics_light.yaml' 2026-05-11T14:30:06,565 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_medicine_light.yaml' 2026-05-11T14:30:06,566 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_physics_light.yaml' 2026-05-11T14:30:06,567 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_computer_security_light.yaml' 2026-05-11T14:30:06,568 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_conceptual_physics_light.yaml' 2026-05-11T14:30:06,569 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_econometrics_light.yaml' 2026-05-11T14:30:06,571 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_electrical_engineering_light.yaml' 2026-05-11T14:30:06,572 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_elementary_mathematics_light.yaml' 2026-05-11T14:30:06,573 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_formal_logic_light.yaml' 2026-05-11T14:30:06,574 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_global_facts_light.yaml' 2026-05-11T14:30:06,575 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_biology_light.yaml' 2026-05-11T14:30:06,577 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_chemistry_light.yaml' 2026-05-11T14:30:06,578 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_computer_science_light.yaml' 2026-05-11T14:30:06,579 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_european_history_light.yaml' 2026-05-11T14:30:06,580 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_geography_light.yaml' 2026-05-11T14:30:06,582 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics_light.yaml' 2026-05-11T14:30:06,583 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics_light.yaml' 2026-05-11T14:30:06,584 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_mathematics_light.yaml' 2026-05-11T14:30:06,585 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_microeconomics_light.yaml' 2026-05-11T14:30:06,586 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_physics_light.yaml' 2026-05-11T14:30:06,588 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_psychology_light.yaml' 2026-05-11T14:30:06,589 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_statistics_light.yaml' 2026-05-11T14:30:06,590 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_us_history_light.yaml' 2026-05-11T14:30:06,591 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_world_history_light.yaml' 2026-05-11T14:30:06,592 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_aging_light.yaml' 2026-05-11T14:30:06,593 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_sexuality_light.yaml' 2026-05-11T14:30:06,594 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_international_law_light.yaml' 2026-05-11T14:30:06,596 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_jurisprudence_light.yaml' 2026-05-11T14:30:06,597 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_light.yaml' 2026-05-11T14:30:06,598 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_logical_fallacies_light.yaml' 2026-05-11T14:30:06,600 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_machine_learning_light.yaml' 2026-05-11T14:30:06,601 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_management_light.yaml' 2026-05-11T14:30:06,602 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_marketing_light.yaml' 2026-05-11T14:30:06,603 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_medical_genetics_light.yaml' 2026-05-11T14:30:06,604 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_miscellaneous_light.yaml' 2026-05-11T14:30:06,606 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_disputes_light.yaml' 2026-05-11T14:30:06,607 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_scenarios_light.yaml' 2026-05-11T14:30:06,608 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_nutrition_light.yaml' 2026-05-11T14:30:06,609 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_philosophy_light.yaml' 2026-05-11T14:30:06,611 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_prehistory_light.yaml' 2026-05-11T14:30:06,612 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_accounting_light.yaml' 2026-05-11T14:30:06,613 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_law_light.yaml' 2026-05-11T14:30:06,614 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_medicine_light.yaml' 2026-05-11T14:30:06,615 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_psychology_light.yaml' 2026-05-11T14:30:06,617 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_public_relations_light.yaml' 2026-05-11T14:30:06,618 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_security_studies_light.yaml' 2026-05-11T14:30:06,619 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_sociology_light.yaml' 2026-05-11T14:30:06,620 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_us_foreign_policy_light.yaml' 2026-05-11T14:30:06,622 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_virology_light.yaml' 2026-05-11T14:30:06,623 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_world_religions_light.yaml' 2026-05-11T14:30:06,624 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/utils.py' 2026-05-11T14:30:06,626 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_leaderboard_arabic_mt_arc_challenge_light.yaml' 2026-05-11T14:30:06,627 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_mt_arc_challenge_light.yaml' 2026-05-11T14:30:06,629 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/utils.py' 2026-05-11T14:30:06,630 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_leaderboard_arabic_mt_arc_easy_light.yaml' 2026-05-11T14:30:06,632 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_mt_arc_easy_light.yaml' 2026-05-11T14:30:06,633 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/utils.py' 2026-05-11T14:30:06,635 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_leaderboard_arabic_mt_boolq_light.yaml' 2026-05-11T14:30:06,636 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_mt_boolq_light.yaml' 2026-05-11T14:30:06,637 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/utils.py' 2026-05-11T14:30:06,639 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arabic_mt_copa_light.yaml' 2026-05-11T14:30:06,640 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arbic_leaderboard_arabic_mt_copa_light.yaml' 2026-05-11T14:30:06,641 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/utils.py' 2026-05-11T14:30:06,643 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_leaderboard_arabic_mt_hellaswag_light.yaml' 2026-05-11T14:30:06,644 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_mt_hellaswag_light.yaml' 2026-05-11T14:30:06,646 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/utils.py' 2026-05-11T14:30:06,647 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_leaderboard_arabic_mt_mmlu_light.yaml' 2026-05-11T14:30:06,649 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_mt_mmlu_light.yaml' 2026-05-11T14:30:06,650 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/utils.py' 2026-05-11T14:30:06,652 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_leaderboard_arabic_mt_openbook_qa_light.yaml' 2026-05-11T14:30:06,653 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_mt_openbook_qa_light.yaml' 2026-05-11T14:30:06,654 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/utils.py' 2026-05-11T14:30:06,655 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_leaderboard_arabic_mt_piqa_light.yaml' 2026-05-11T14:30:06,657 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_mt_piqa_light.yaml' 2026-05-11T14:30:06,658 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/utils.py' 2026-05-11T14:30:06,659 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_leaderboard_arabic_mt_race_light.yaml' 2026-05-11T14:30:06,661 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_mt_race_light.yaml' 2026-05-11T14:30:06,662 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/utils.py' 2026-05-11T14:30:06,664 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_leaderboard_arabic_mt_sciq_light.yaml' 2026-05-11T14:30:06,665 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_mt_sciq_light.yaml' 2026-05-11T14:30:06,666 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/utils.py' 2026-05-11T14:30:06,668 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_leaderboard_arabic_mt_toxigen_light.yaml' 2026-05-11T14:30:06,669 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_mt_toxigen_light.yaml' 2026-05-11T14:30:06,670 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/utils.py' 2026-05-11T14:30:06,673 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Algeria_light.yaml' 2026-05-11T14:30:06,674 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Ancient_Egypt_light.yaml' 2026-05-11T14:30:06,675 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arab_Empire_light.yaml' 2026-05-11T14:30:06,677 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Architecture_light.yaml' 2026-05-11T14:30:06,678 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Art_light.yaml' 2026-05-11T14:30:06,679 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Astronomy_light.yaml' 2026-05-11T14:30:06,680 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Calligraphy_light.yaml' 2026-05-11T14:30:06,682 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ceremony_light.yaml' 2026-05-11T14:30:06,683 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Clothing_light.yaml' 2026-05-11T14:30:06,684 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Culture_light.yaml' 2026-05-11T14:30:06,685 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Food_light.yaml' 2026-05-11T14:30:06,687 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Funeral_light.yaml' 2026-05-11T14:30:06,688 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Geography_light.yaml' 2026-05-11T14:30:06,689 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_History_light.yaml' 2026-05-11T14:30:06,690 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Language_Origin_light.yaml' 2026-05-11T14:30:06,691 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Literature_light.yaml' 2026-05-11T14:30:06,692 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Math_light.yaml' 2026-05-11T14:30:06,693 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Medicine_light.yaml' 2026-05-11T14:30:06,695 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Music_light.yaml' 2026-05-11T14:30:06,696 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ornament_light.yaml' 2026-05-11T14:30:06,697 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Philosophy_light.yaml' 2026-05-11T14:30:06,698 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry_light.yaml' 2026-05-11T14:30:06,699 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Wedding_light.yaml' 2026-05-11T14:30:06,701 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Bahrain_light.yaml' 2026-05-11T14:30:06,702 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Comoros_light.yaml' 2026-05-11T14:30:06,703 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Egypt_modern_light.yaml' 2026-05-11T14:30:06,704 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromAncientEgypt_light.yaml' 2026-05-11T14:30:06,706 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromByzantium_light.yaml' 2026-05-11T14:30:06,707 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromChina_light.yaml' 2026-05-11T14:30:06,708 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromGreece_light.yaml' 2026-05-11T14:30:06,709 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromIslam_light.yaml' 2026-05-11T14:30:06,710 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromPersia_light.yaml' 2026-05-11T14:30:06,712 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromRome_light.yaml' 2026-05-11T14:30:06,713 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Iraq_light.yaml' 2026-05-11T14:30:06,714 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_Education_light.yaml' 2026-05-11T14:30:06,715 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_branches_and_schools_light.yaml' 2026-05-11T14:30:06,717 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islamic_law_system_light.yaml' 2026-05-11T14:30:06,718 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Jordan_light.yaml' 2026-05-11T14:30:06,719 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Kuwait_light.yaml' 2026-05-11T14:30:06,720 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Lebanon_light.yaml' 2026-05-11T14:30:06,721 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Libya_light.yaml' 2026-05-11T14:30:06,723 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mauritania_light.yaml' 2026-05-11T14:30:06,724 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mesopotamia_civilization_light.yaml' 2026-05-11T14:30:06,725 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Morocco_light.yaml' 2026-05-11T14:30:06,726 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Oman_light.yaml' 2026-05-11T14:30:06,727 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Palestine_light.yaml' 2026-05-11T14:30:06,728 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Qatar_light.yaml' 2026-05-11T14:30:06,729 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Saudi_Arabia_light.yaml' 2026-05-11T14:30:06,731 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Somalia_light.yaml' 2026-05-11T14:30:06,732 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Sudan_light.yaml' 2026-05-11T14:30:06,733 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Syria_light.yaml' 2026-05-11T14:30:06,734 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Tunisia_light.yaml' 2026-05-11T14:30:06,735 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_United_Arab_Emirates_light.yaml' 2026-05-11T14:30:06,737 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Yemen_light.yaml' 2026-05-11T14:30:06,738 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_communication_light.yaml' 2026-05-11T14:30:06,739 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_computer_and_phone_light.yaml' 2026-05-11T14:30:06,740 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_daily_life_light.yaml' 2026-05-11T14:30:06,742 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_entertainment_light.yaml' 2026-05-11T14:30:06,743 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_light.yaml' 2026-05-11T14:30:06,744 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/utils.py' 2026-05-11T14:30:06,747 adding 'lm_eval/tasks/arabicmmlu/README.md' 2026-05-11T14:30:06,748 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu.yaml' 2026-05-11T14:30:06,749 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_humanities.yaml' 2026-05-11T14:30:06,751 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_language.yaml' 2026-05-11T14:30:06,752 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_other.yaml' 2026-05-11T14:30:06,753 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_social_science.yaml' 2026-05-11T14:30:06,754 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_stem.yaml' 2026-05-11T14:30:06,755 adding 'lm_eval/tasks/arabicmmlu/_default_arabicmmlu_template_yaml' 2026-05-11T14:30:06,757 adding 'lm_eval/tasks/arabicmmlu/_generate_configs.py' 2026-05-11T14:30:06,758 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_accounting_university.yaml' 2026-05-11T14:30:06,759 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml' 2026-05-11T14:30:06,760 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml' 2026-05-11T14:30:06,761 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_high_school.yaml' 2026-05-11T14:30:06,762 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_middle_school.yaml' 2026-05-11T14:30:06,763 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_primary_school.yaml' 2026-05-11T14:30:06,764 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_biology_high_school.yaml' 2026-05-11T14:30:06,765 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_civics_high_school.yaml' 2026-05-11T14:30:06,766 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_civics_middle_school.yaml' 2026-05-11T14:30:06,768 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_high_school.yaml' 2026-05-11T14:30:06,769 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_middle_school.yaml' 2026-05-11T14:30:06,770 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_primary_school.yaml' 2026-05-11T14:30:06,771 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_university.yaml' 2026-05-11T14:30:06,772 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml' 2026-05-11T14:30:06,773 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_economics_high_school.yaml' 2026-05-11T14:30:06,774 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_economics_middle_school.yaml' 2026-05-11T14:30:06,775 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_economics_university.yaml' 2026-05-11T14:30:06,777 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml' 2026-05-11T14:30:06,778 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_middle_school.yaml' 2026-05-11T14:30:06,779 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_primary_school.yaml' 2026-05-11T14:30:06,780 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_geography_high_school.yaml' 2026-05-11T14:30:06,781 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_geography_middle_school.yaml' 2026-05-11T14:30:06,783 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_geography_primary_school.yaml' 2026-05-11T14:30:06,784 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_history_high_school.yaml' 2026-05-11T14:30:06,785 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_history_middle_school.yaml' 2026-05-11T14:30:06,786 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_history_primary_school.yaml' 2026-05-11T14:30:06,787 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml' 2026-05-11T14:30:06,789 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_high_school.yaml' 2026-05-11T14:30:06,790 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_middle_school.yaml' 2026-05-11T14:30:06,791 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_primary_school.yaml' 2026-05-11T14:30:06,792 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_law_professional.yaml' 2026-05-11T14:30:06,793 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_management_university.yaml' 2026-05-11T14:30:06,795 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_math_primary_school.yaml' 2026-05-11T14:30:06,796 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_middle_school.yaml' 2026-05-11T14:30:06,797 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_primary_school.yaml' 2026-05-11T14:30:06,798 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_philosophy_high_school.yaml' 2026-05-11T14:30:06,800 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_physics_high_school.yaml' 2026-05-11T14:30:06,801 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_political_science_university.yaml' 2026-05-11T14:30:06,802 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_middle_school.yaml' 2026-05-11T14:30:06,803 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_primary_school.yaml' 2026-05-11T14:30:06,804 adding 'lm_eval/tasks/arabicmmlu/utils.py' 2026-05-11T14:30:06,806 adding 'lm_eval/tasks/aradice/README.md' 2026-05-11T14:30:06,808 adding 'lm_eval/tasks/aradice/aradice.yaml' 2026-05-11T14:30:06,811 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU.yaml' 2026-05-11T14:30:06,812 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_history.yaml' 2026-05-11T14:30:06,813 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,814 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml' 2026-05-11T14:30:06,815 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml' 2026-05-11T14:30:06,816 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_civics.yaml' 2026-05-11T14:30:06,817 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_economics.yaml' 2026-05-11T14:30:06,818 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_geography.yaml' 2026-05-11T14:30:06,819 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_biology.yaml' 2026-05-11T14:30:06,821 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml' 2026-05-11T14:30:06,822 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_physics.yaml' 2026-05-11T14:30:06,823 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_history.yaml' 2026-05-11T14:30:06,824 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,825 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml' 2026-05-11T14:30:06,826 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml' 2026-05-11T14:30:06,827 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml' 2026-05-11T14:30:06,829 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml' 2026-05-11T14:30:06,830 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml' 2026-05-11T14:30:06,831 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml' 2026-05-11T14:30:06,832 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml' 2026-05-11T14:30:06,833 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml' 2026-05-11T14:30:06,834 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,836 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml' 2026-05-11T14:30:06,837 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml' 2026-05-11T14:30:06,838 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_driving-test.yaml' 2026-05-11T14:30:06,839 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml' 2026-05-11T14:30:06,840 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_history.yaml' 2026-05-11T14:30:06,841 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,843 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml' 2026-05-11T14:30:06,844 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml' 2026-05-11T14:30:06,845 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml' 2026-05-11T14:30:06,846 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml' 2026-05-11T14:30:06,847 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml' 2026-05-11T14:30:06,849 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_math.yaml' 2026-05-11T14:30:06,850 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml' 2026-05-11T14:30:06,851 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_prof_humanities_law.yaml' 2026-05-11T14:30:06,852 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_other_management.yaml' 2026-05-11T14:30:06,853 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml' 2026-05-11T14:30:06,854 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml' 2026-05-11T14:30:06,855 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml' 2026-05-11T14:30:06,856 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml' 2026-05-11T14:30:06,858 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/_default_template_yaml' 2026-05-11T14:30:06,859 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/metrics.py' 2026-05-11T14:30:06,860 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/utils.py' 2026-05-11T14:30:06,863 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU.yaml' 2026-05-11T14:30:06,864 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_history.yaml' 2026-05-11T14:30:06,865 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,866 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml' 2026-05-11T14:30:06,867 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml' 2026-05-11T14:30:06,868 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_civics.yaml' 2026-05-11T14:30:06,870 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_economics.yaml' 2026-05-11T14:30:06,871 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_geography.yaml' 2026-05-11T14:30:06,872 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_biology.yaml' 2026-05-11T14:30:06,873 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml' 2026-05-11T14:30:06,874 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_physics.yaml' 2026-05-11T14:30:06,875 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_history.yaml' 2026-05-11T14:30:06,877 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,878 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml' 2026-05-11T14:30:06,879 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml' 2026-05-11T14:30:06,880 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml' 2026-05-11T14:30:06,881 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml' 2026-05-11T14:30:06,885 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml' 2026-05-11T14:30:06,886 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml' 2026-05-11T14:30:06,887 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml' 2026-05-11T14:30:06,888 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml' 2026-05-11T14:30:06,890 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,891 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml' 2026-05-11T14:30:06,892 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml' 2026-05-11T14:30:06,893 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_driving-test.yaml' 2026-05-11T14:30:06,894 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml' 2026-05-11T14:30:06,895 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_history.yaml' 2026-05-11T14:30:06,897 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml' 2026-05-11T14:30:06,898 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml' 2026-05-11T14:30:06,899 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml' 2026-05-11T14:30:06,900 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml' 2026-05-11T14:30:06,902 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml' 2026-05-11T14:30:06,903 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml' 2026-05-11T14:30:06,904 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_math.yaml' 2026-05-11T14:30:06,905 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml' 2026-05-11T14:30:06,906 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_prof_humanities_law.yaml' 2026-05-11T14:30:06,907 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_other_management.yaml' 2026-05-11T14:30:06,909 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml' 2026-05-11T14:30:06,910 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml' 2026-05-11T14:30:06,911 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml' 2026-05-11T14:30:06,912 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml' 2026-05-11T14:30:06,913 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/_default_template_yaml' 2026-05-11T14:30:06,914 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/metrics.py' 2026-05-11T14:30:06,916 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/utils.py' 2026-05-11T14:30:06,918 adding 'lm_eval/tasks/aradice/boolq/EGY/boolq_egy.yaml' 2026-05-11T14:30:06,919 adding 'lm_eval/tasks/aradice/boolq/EGY/metrics.py' 2026-05-11T14:30:06,921 adding 'lm_eval/tasks/aradice/boolq/EGY/utils.py' 2026-05-11T14:30:06,922 adding 'lm_eval/tasks/aradice/boolq/ENG/boolq_eng.yaml' 2026-05-11T14:30:06,923 adding 'lm_eval/tasks/aradice/boolq/ENG/metrics.py' 2026-05-11T14:30:06,925 adding 'lm_eval/tasks/aradice/boolq/ENG/utils.py' 2026-05-11T14:30:06,926 adding 'lm_eval/tasks/aradice/boolq/LEV/boolq_lev.yaml' 2026-05-11T14:30:06,928 adding 'lm_eval/tasks/aradice/boolq/LEV/metrics.py' 2026-05-11T14:30:06,929 adding 'lm_eval/tasks/aradice/boolq/LEV/utils.py' 2026-05-11T14:30:06,931 adding 'lm_eval/tasks/aradice/boolq/MSA/boolq_msa.yaml' 2026-05-11T14:30:06,932 adding 'lm_eval/tasks/aradice/boolq/MSA/metrics.py' 2026-05-11T14:30:06,933 adding 'lm_eval/tasks/aradice/boolq/MSA/utils.py' 2026-05-11T14:30:06,935 adding 'lm_eval/tasks/aradice/cultural-benchmark/egypt.yaml' 2026-05-11T14:30:06,936 adding 'lm_eval/tasks/aradice/cultural-benchmark/jordan.yaml' 2026-05-11T14:30:06,937 adding 'lm_eval/tasks/aradice/cultural-benchmark/lebanon.yaml' 2026-05-11T14:30:06,939 adding 'lm_eval/tasks/aradice/cultural-benchmark/metrics.py' 2026-05-11T14:30:06,940 adding 'lm_eval/tasks/aradice/cultural-benchmark/palestine.yaml' 2026-05-11T14:30:06,941 adding 'lm_eval/tasks/aradice/cultural-benchmark/qatar.yaml' 2026-05-11T14:30:06,942 adding 'lm_eval/tasks/aradice/cultural-benchmark/syria.yaml' 2026-05-11T14:30:06,943 adding 'lm_eval/tasks/aradice/cultural-benchmark/utils.py' 2026-05-11T14:30:06,945 adding 'lm_eval/tasks/aradice/openbookqa/metrics.py' 2026-05-11T14:30:06,946 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_egy.yaml' 2026-05-11T14:30:06,947 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_eng.yaml' 2026-05-11T14:30:06,948 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_lev.yaml' 2026-05-11T14:30:06,949 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_msa.yaml' 2026-05-11T14:30:06,950 adding 'lm_eval/tasks/aradice/openbookqa/utils.py' 2026-05-11T14:30:06,952 adding 'lm_eval/tasks/aradice/piqa/metrics.py' 2026-05-11T14:30:06,953 adding 'lm_eval/tasks/aradice/piqa/piqa_egy.yaml' 2026-05-11T14:30:06,954 adding 'lm_eval/tasks/aradice/piqa/piqa_eng.yaml' 2026-05-11T14:30:06,955 adding 'lm_eval/tasks/aradice/piqa/piqa_lev.yaml' 2026-05-11T14:30:06,957 adding 'lm_eval/tasks/aradice/piqa/piqa_msa.yaml' 2026-05-11T14:30:06,958 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/metrics.py' 2026-05-11T14:30:06,960 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_egy.yaml' 2026-05-11T14:30:06,961 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_eng.yaml' 2026-05-11T14:30:06,962 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_lev.yaml' 2026-05-11T14:30:06,963 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_msa.yaml' 2026-05-11T14:30:06,965 adding 'lm_eval/tasks/aradice/winogrande/metrics.py' 2026-05-11T14:30:06,966 adding 'lm_eval/tasks/aradice/winogrande/utils.py' 2026-05-11T14:30:06,967 adding 'lm_eval/tasks/aradice/winogrande/winogrande_egy.yaml' 2026-05-11T14:30:06,968 adding 'lm_eval/tasks/aradice/winogrande/winogrande_eng.yaml' 2026-05-11T14:30:06,970 adding 'lm_eval/tasks/aradice/winogrande/winogrande_lev.yaml' 2026-05-11T14:30:06,971 adding 'lm_eval/tasks/aradice/winogrande/winogrande_msa.yaml' 2026-05-11T14:30:06,973 adding 'lm_eval/tasks/arc/README.md' 2026-05-11T14:30:06,974 adding 'lm_eval/tasks/arc/arc_challenge.yaml' 2026-05-11T14:30:06,975 adding 'lm_eval/tasks/arc/arc_challenge_chat.yaml' 2026-05-11T14:30:06,976 adding 'lm_eval/tasks/arc/arc_easy.yaml' 2026-05-11T14:30:06,978 adding 'lm_eval/tasks/arc_mt/README.md' 2026-05-11T14:30:06,979 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml' 2026-05-11T14:30:06,980 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml' 2026-05-11T14:30:06,982 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml' 2026-05-11T14:30:06,983 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml' 2026-05-11T14:30:06,984 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml' 2026-05-11T14:30:06,985 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml' 2026-05-11T14:30:06,986 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml' 2026-05-11T14:30:06,988 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml' 2026-05-11T14:30:06,989 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml' 2026-05-11T14:30:06,990 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml' 2026-05-11T14:30:06,991 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml' 2026-05-11T14:30:06,992 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml' 2026-05-11T14:30:06,994 adding 'lm_eval/tasks/arithmetic/README.md' 2026-05-11T14:30:06,995 adding 'lm_eval/tasks/arithmetic/arithmetic_1dc.yaml' 2026-05-11T14:30:06,996 adding 'lm_eval/tasks/arithmetic/arithmetic_2da.yaml' 2026-05-11T14:30:06,998 adding 'lm_eval/tasks/arithmetic/arithmetic_2dm.yaml' 2026-05-11T14:30:07,000 adding 'lm_eval/tasks/arithmetic/arithmetic_2ds.yaml' 2026-05-11T14:30:07,001 adding 'lm_eval/tasks/arithmetic/arithmetic_3da.yaml' 2026-05-11T14:30:07,002 adding 'lm_eval/tasks/arithmetic/arithmetic_3ds.yaml' 2026-05-11T14:30:07,003 adding 'lm_eval/tasks/arithmetic/arithmetic_4da.yaml' 2026-05-11T14:30:07,004 adding 'lm_eval/tasks/arithmetic/arithmetic_4ds.yaml' 2026-05-11T14:30:07,005 adding 'lm_eval/tasks/arithmetic/arithmetic_5da.yaml' 2026-05-11T14:30:07,007 adding 'lm_eval/tasks/arithmetic/arithmetic_5ds.yaml' 2026-05-11T14:30:07,008 adding 'lm_eval/tasks/asdiv/README.md' 2026-05-11T14:30:07,010 adding 'lm_eval/tasks/asdiv/asdiv-cot-llama.yaml' 2026-05-11T14:30:07,011 adding 'lm_eval/tasks/asdiv/default.yaml' 2026-05-11T14:30:07,013 adding 'lm_eval/tasks/babi/README.md' 2026-05-11T14:30:07,014 adding 'lm_eval/tasks/babi/babi.yaml' 2026-05-11T14:30:07,016 adding 'lm_eval/tasks/babilong/README.md' 2026-05-11T14:30:07,017 adding 'lm_eval/tasks/babilong/_babilong_common_yaml' 2026-05-11T14:30:07,018 adding 'lm_eval/tasks/babilong/babilong.yaml' 2026-05-11T14:30:07,019 adding 'lm_eval/tasks/babilong/babilong_longctx.yaml' 2026-05-11T14:30:07,020 adding 'lm_eval/tasks/babilong/babilong_qa1.yaml' 2026-05-11T14:30:07,022 adding 'lm_eval/tasks/babilong/babilong_qa10.yaml' 2026-05-11T14:30:07,023 adding 'lm_eval/tasks/babilong/babilong_qa11.yaml' 2026-05-11T14:30:07,024 adding 'lm_eval/tasks/babilong/babilong_qa12.yaml' 2026-05-11T14:30:07,025 adding 'lm_eval/tasks/babilong/babilong_qa13.yaml' 2026-05-11T14:30:07,027 adding 'lm_eval/tasks/babilong/babilong_qa14.yaml' 2026-05-11T14:30:07,028 adding 'lm_eval/tasks/babilong/babilong_qa15.yaml' 2026-05-11T14:30:07,029 adding 'lm_eval/tasks/babilong/babilong_qa16.yaml' 2026-05-11T14:30:07,030 adding 'lm_eval/tasks/babilong/babilong_qa17.yaml' 2026-05-11T14:30:07,031 adding 'lm_eval/tasks/babilong/babilong_qa18.yaml' 2026-05-11T14:30:07,033 adding 'lm_eval/tasks/babilong/babilong_qa19.yaml' 2026-05-11T14:30:07,034 adding 'lm_eval/tasks/babilong/babilong_qa2.yaml' 2026-05-11T14:30:07,035 adding 'lm_eval/tasks/babilong/babilong_qa20.yaml' 2026-05-11T14:30:07,036 adding 'lm_eval/tasks/babilong/babilong_qa3.yaml' 2026-05-11T14:30:07,037 adding 'lm_eval/tasks/babilong/babilong_qa4.yaml' 2026-05-11T14:30:07,039 adding 'lm_eval/tasks/babilong/babilong_qa5.yaml' 2026-05-11T14:30:07,040 adding 'lm_eval/tasks/babilong/babilong_qa6.yaml' 2026-05-11T14:30:07,041 adding 'lm_eval/tasks/babilong/babilong_qa7.yaml' 2026-05-11T14:30:07,042 adding 'lm_eval/tasks/babilong/babilong_qa8.yaml' 2026-05-11T14:30:07,044 adding 'lm_eval/tasks/babilong/babilong_qa9.yaml' 2026-05-11T14:30:07,045 adding 'lm_eval/tasks/babilong/common_utils.py' 2026-05-11T14:30:07,047 adding 'lm_eval/tasks/bangla/README.md' 2026-05-11T14:30:07,049 adding 'lm_eval/tasks/bangla/bangla_boolqa.yaml' 2026-05-11T14:30:07,050 adding 'lm_eval/tasks/bangla/bangla_commonsenseqa.yaml' 2026-05-11T14:30:07,051 adding 'lm_eval/tasks/bangla/bangla_mmlu.yaml' 2026-05-11T14:30:07,052 adding 'lm_eval/tasks/bangla/bangla_openbookqa.yaml' 2026-05-11T14:30:07,054 adding 'lm_eval/tasks/bangla/bangla_piqa.yaml' 2026-05-11T14:30:07,056 adding 'lm_eval/tasks/basque_bench/README.md' 2026-05-11T14:30:07,057 adding 'lm_eval/tasks/basque_bench/arc_eu_challenge.yaml' 2026-05-11T14:30:07,059 adding 'lm_eval/tasks/basque_bench/arc_eu_easy.yaml' 2026-05-11T14:30:07,060 adding 'lm_eval/tasks/basque_bench/basque_bench.yaml' 2026-05-11T14:30:07,061 adding 'lm_eval/tasks/basque_bench/mgsm_direct_eu.yaml' 2026-05-11T14:30:07,062 adding 'lm_eval/tasks/basque_bench/mgsm_native_cot_eu.yaml' 2026-05-11T14:30:07,064 adding 'lm_eval/tasks/basque_bench/paws_eu.yaml' 2026-05-11T14:30:07,065 adding 'lm_eval/tasks/basque_bench/piqa_eu.yaml' 2026-05-11T14:30:07,066 adding 'lm_eval/tasks/basque_bench/utils.py' 2026-05-11T14:30:07,068 adding 'lm_eval/tasks/basque_bench/wnli_eu.yaml' 2026-05-11T14:30:07,069 adding 'lm_eval/tasks/basque_bench/xcopa_eu.yaml' 2026-05-11T14:30:07,071 adding 'lm_eval/tasks/basque_bench/flores_eu/_flores_common_yaml' 2026-05-11T14:30:07,073 adding 'lm_eval/tasks/basque_bench/flores_eu/create_yamls_flores_eu.py' 2026-05-11T14:30:07,074 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_ca-eu.yaml' 2026-05-11T14:30:07,075 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_de-eu.yaml' 2026-05-11T14:30:07,076 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_en-eu.yaml' 2026-05-11T14:30:07,077 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_es-eu.yaml' 2026-05-11T14:30:07,078 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-ca.yaml' 2026-05-11T14:30:07,079 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-de.yaml' 2026-05-11T14:30:07,080 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-en.yaml' 2026-05-11T14:30:07,081 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-es.yaml' 2026-05-11T14:30:07,082 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-fr.yaml' 2026-05-11T14:30:07,084 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-gl.yaml' 2026-05-11T14:30:07,085 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-it.yaml' 2026-05-11T14:30:07,086 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-pt.yaml' 2026-05-11T14:30:07,087 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu.yaml' 2026-05-11T14:30:07,088 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_fr-eu.yaml' 2026-05-11T14:30:07,089 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_gl-eu.yaml' 2026-05-11T14:30:07,090 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_it-eu.yaml' 2026-05-11T14:30:07,091 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_pt-eu.yaml' 2026-05-11T14:30:07,093 adding 'lm_eval/tasks/basqueglue/README.md' 2026-05-11T14:30:07,094 adding 'lm_eval/tasks/basqueglue/bec.yaml' 2026-05-11T14:30:07,096 adding 'lm_eval/tasks/basqueglue/bhtc.yaml' 2026-05-11T14:30:07,098 adding 'lm_eval/tasks/basqueglue/coref.yaml' 2026-05-11T14:30:07,099 adding 'lm_eval/tasks/basqueglue/qnli.yaml' 2026-05-11T14:30:07,101 adding 'lm_eval/tasks/basqueglue/utils.py' 2026-05-11T14:30:07,102 adding 'lm_eval/tasks/basqueglue/vaxx.yaml' 2026-05-11T14:30:07,103 adding 'lm_eval/tasks/basqueglue/wic.yaml' 2026-05-11T14:30:07,105 adding 'lm_eval/tasks/bbh/README.md' 2026-05-11T14:30:07,107 adding 'lm_eval/tasks/bbh/_generate_configs.py' 2026-05-11T14:30:07,109 adding 'lm_eval/tasks/bbh/cot_fewshot/_bbh.yaml' 2026-05-11T14:30:07,111 adding 'lm_eval/tasks/bbh/cot_fewshot/_bbh_cot_fewshot.yaml' 2026-05-11T14:30:07,112 adding 'lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml' 2026-05-11T14:30:07,114 adding 'lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml' 2026-05-11T14:30:07,115 adding 'lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml' 2026-05-11T14:30:07,117 adding 'lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml' 2026-05-11T14:30:07,119 adding 'lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml' 2026-05-11T14:30:07,120 adding 'lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml' 2026-05-11T14:30:07,122 adding 'lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml' 2026-05-11T14:30:07,124 adding 'lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml' 2026-05-11T14:30:07,126 adding 'lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml' 2026-05-11T14:30:07,127 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml' 2026-05-11T14:30:07,129 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml' 2026-05-11T14:30:07,130 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml' 2026-05-11T14:30:07,132 adding 'lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml' 2026-05-11T14:30:07,134 adding 'lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml' 2026-05-11T14:30:07,135 adding 'lm_eval/tasks/bbh/cot_fewshot/navigate.yaml' 2026-05-11T14:30:07,137 adding 'lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml' 2026-05-11T14:30:07,139 adding 'lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml' 2026-05-11T14:30:07,140 adding 'lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml' 2026-05-11T14:30:07,142 adding 'lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml' 2026-05-11T14:30:07,144 adding 'lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml' 2026-05-11T14:30:07,146 adding 'lm_eval/tasks/bbh/cot_fewshot/snarks.yaml' 2026-05-11T14:30:07,147 adding 'lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml' 2026-05-11T14:30:07,149 adding 'lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml' 2026-05-11T14:30:07,151 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T14:30:07,152 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T14:30:07,154 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T14:30:07,155 adding 'lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml' 2026-05-11T14:30:07,157 adding 'lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml' 2026-05-11T14:30:07,160 adding 'lm_eval/tasks/bbh/cot_zeroshot/_bbh_cot_zeroshot.yaml' 2026-05-11T14:30:07,161 adding 'lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml' 2026-05-11T14:30:07,162 adding 'lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml' 2026-05-11T14:30:07,164 adding 'lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml' 2026-05-11T14:30:07,165 adding 'lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml' 2026-05-11T14:30:07,167 adding 'lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml' 2026-05-11T14:30:07,168 adding 'lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml' 2026-05-11T14:30:07,170 adding 'lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml' 2026-05-11T14:30:07,171 adding 'lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml' 2026-05-11T14:30:07,173 adding 'lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml' 2026-05-11T14:30:07,174 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml' 2026-05-11T14:30:07,176 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml' 2026-05-11T14:30:07,177 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml' 2026-05-11T14:30:07,179 adding 'lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml' 2026-05-11T14:30:07,180 adding 'lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml' 2026-05-11T14:30:07,182 adding 'lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml' 2026-05-11T14:30:07,183 adding 'lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml' 2026-05-11T14:30:07,185 adding 'lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml' 2026-05-11T14:30:07,186 adding 'lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml' 2026-05-11T14:30:07,187 adding 'lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml' 2026-05-11T14:30:07,189 adding 'lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml' 2026-05-11T14:30:07,190 adding 'lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml' 2026-05-11T14:30:07,192 adding 'lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml' 2026-05-11T14:30:07,193 adding 'lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml' 2026-05-11T14:30:07,195 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T14:30:07,197 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T14:30:07,198 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T14:30:07,200 adding 'lm_eval/tasks/bbh/cot_zeroshot/utils.py' 2026-05-11T14:30:07,201 adding 'lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml' 2026-05-11T14:30:07,202 adding 'lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml' 2026-05-11T14:30:07,205 adding 'lm_eval/tasks/bbh/fewshot/_bbh_fewshot.yaml' 2026-05-11T14:30:07,206 adding 'lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml' 2026-05-11T14:30:07,207 adding 'lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml' 2026-05-11T14:30:07,208 adding 'lm_eval/tasks/bbh/fewshot/causal_judgement.yaml' 2026-05-11T14:30:07,210 adding 'lm_eval/tasks/bbh/fewshot/date_understanding.yaml' 2026-05-11T14:30:07,211 adding 'lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml' 2026-05-11T14:30:07,212 adding 'lm_eval/tasks/bbh/fewshot/dyck_languages.yaml' 2026-05-11T14:30:07,214 adding 'lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml' 2026-05-11T14:30:07,215 adding 'lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml' 2026-05-11T14:30:07,216 adding 'lm_eval/tasks/bbh/fewshot/hyperbaton.yaml' 2026-05-11T14:30:07,217 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml' 2026-05-11T14:30:07,218 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml' 2026-05-11T14:30:07,219 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml' 2026-05-11T14:30:07,220 adding 'lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml' 2026-05-11T14:30:07,222 adding 'lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml' 2026-05-11T14:30:07,223 adding 'lm_eval/tasks/bbh/fewshot/navigate.yaml' 2026-05-11T14:30:07,224 adding 'lm_eval/tasks/bbh/fewshot/object_counting.yaml' 2026-05-11T14:30:07,225 adding 'lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml' 2026-05-11T14:30:07,226 adding 'lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml' 2026-05-11T14:30:07,228 adding 'lm_eval/tasks/bbh/fewshot/ruin_names.yaml' 2026-05-11T14:30:07,229 adding 'lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml' 2026-05-11T14:30:07,230 adding 'lm_eval/tasks/bbh/fewshot/snarks.yaml' 2026-05-11T14:30:07,231 adding 'lm_eval/tasks/bbh/fewshot/sports_understanding.yaml' 2026-05-11T14:30:07,233 adding 'lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml' 2026-05-11T14:30:07,234 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T14:30:07,235 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T14:30:07,236 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T14:30:07,238 adding 'lm_eval/tasks/bbh/fewshot/web_of_lies.yaml' 2026-05-11T14:30:07,239 adding 'lm_eval/tasks/bbh/fewshot/word_sorting.yaml' 2026-05-11T14:30:07,241 adding 'lm_eval/tasks/bbh/zeroshot/_bbh_zeroshot.yaml' 2026-05-11T14:30:07,242 adding 'lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml' 2026-05-11T14:30:07,243 adding 'lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml' 2026-05-11T14:30:07,245 adding 'lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml' 2026-05-11T14:30:07,246 adding 'lm_eval/tasks/bbh/zeroshot/date_understanding.yaml' 2026-05-11T14:30:07,247 adding 'lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml' 2026-05-11T14:30:07,248 adding 'lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml' 2026-05-11T14:30:07,249 adding 'lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml' 2026-05-11T14:30:07,251 adding 'lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml' 2026-05-11T14:30:07,252 adding 'lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml' 2026-05-11T14:30:07,253 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml' 2026-05-11T14:30:07,254 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml' 2026-05-11T14:30:07,256 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml' 2026-05-11T14:30:07,257 adding 'lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml' 2026-05-11T14:30:07,258 adding 'lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml' 2026-05-11T14:30:07,259 adding 'lm_eval/tasks/bbh/zeroshot/navigate.yaml' 2026-05-11T14:30:07,260 adding 'lm_eval/tasks/bbh/zeroshot/object_counting.yaml' 2026-05-11T14:30:07,262 adding 'lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml' 2026-05-11T14:30:07,263 adding 'lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml' 2026-05-11T14:30:07,264 adding 'lm_eval/tasks/bbh/zeroshot/ruin_names.yaml' 2026-05-11T14:30:07,265 adding 'lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml' 2026-05-11T14:30:07,267 adding 'lm_eval/tasks/bbh/zeroshot/snarks.yaml' 2026-05-11T14:30:07,268 adding 'lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml' 2026-05-11T14:30:07,269 adding 'lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml' 2026-05-11T14:30:07,270 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T14:30:07,271 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T14:30:07,273 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T14:30:07,274 adding 'lm_eval/tasks/bbh/zeroshot/utils.py' 2026-05-11T14:30:07,275 adding 'lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml' 2026-05-11T14:30:07,276 adding 'lm_eval/tasks/bbh/zeroshot/word_sorting.yaml' 2026-05-11T14:30:07,278 adding 'lm_eval/tasks/bbq/README.md' 2026-05-11T14:30:07,279 adding 'lm_eval/tasks/bbq/bbq_generate.yaml' 2026-05-11T14:30:07,280 adding 'lm_eval/tasks/bbq/bbq_generate_ambig.yaml' 2026-05-11T14:30:07,281 adding 'lm_eval/tasks/bbq/bbq_generate_disambig.yaml' 2026-05-11T14:30:07,283 adding 'lm_eval/tasks/bbq/bbq_multiple_choice.yaml' 2026-05-11T14:30:07,284 adding 'lm_eval/tasks/bbq/bbq_multiple_choice_ambig.yaml' 2026-05-11T14:30:07,285 adding 'lm_eval/tasks/bbq/bbq_multiple_choice_disambig.yaml' 2026-05-11T14:30:07,287 adding 'lm_eval/tasks/bbq/utils.py' 2026-05-11T14:30:07,289 adding 'lm_eval/tasks/bear/README.md' 2026-05-11T14:30:07,290 adding 'lm_eval/tasks/bear/bear.yaml' 2026-05-11T14:30:07,291 adding 'lm_eval/tasks/bear/bear_big.yaml' 2026-05-11T14:30:07,295 adding 'lm_eval/tasks/belebele/README.md' 2026-05-11T14:30:07,296 adding 'lm_eval/tasks/belebele/_belebele.yaml' 2026-05-11T14:30:07,297 adding 'lm_eval/tasks/belebele/_default_template_yaml' 2026-05-11T14:30:07,299 adding 'lm_eval/tasks/belebele/_generate_configs.py' 2026-05-11T14:30:07,300 adding 'lm_eval/tasks/belebele/belebele_acm_Arab.yaml' 2026-05-11T14:30:07,301 adding 'lm_eval/tasks/belebele/belebele_afr_Latn.yaml' 2026-05-11T14:30:07,302 adding 'lm_eval/tasks/belebele/belebele_als_Latn.yaml' 2026-05-11T14:30:07,303 adding 'lm_eval/tasks/belebele/belebele_amh_Ethi.yaml' 2026-05-11T14:30:07,305 adding 'lm_eval/tasks/belebele/belebele_apc_Arab.yaml' 2026-05-11T14:30:07,306 adding 'lm_eval/tasks/belebele/belebele_arb_Arab.yaml' 2026-05-11T14:30:07,307 adding 'lm_eval/tasks/belebele/belebele_arb_Latn.yaml' 2026-05-11T14:30:07,308 adding 'lm_eval/tasks/belebele/belebele_ars_Arab.yaml' 2026-05-11T14:30:07,309 adding 'lm_eval/tasks/belebele/belebele_ary_Arab.yaml' 2026-05-11T14:30:07,310 adding 'lm_eval/tasks/belebele/belebele_arz_Arab.yaml' 2026-05-11T14:30:07,311 adding 'lm_eval/tasks/belebele/belebele_asm_Beng.yaml' 2026-05-11T14:30:07,313 adding 'lm_eval/tasks/belebele/belebele_azj_Latn.yaml' 2026-05-11T14:30:07,314 adding 'lm_eval/tasks/belebele/belebele_bam_Latn.yaml' 2026-05-11T14:30:07,315 adding 'lm_eval/tasks/belebele/belebele_ben_Beng.yaml' 2026-05-11T14:30:07,316 adding 'lm_eval/tasks/belebele/belebele_ben_Latn.yaml' 2026-05-11T14:30:07,317 adding 'lm_eval/tasks/belebele/belebele_bod_Tibt.yaml' 2026-05-11T14:30:07,318 adding 'lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml' 2026-05-11T14:30:07,323 adding 'lm_eval/tasks/belebele/belebele_cat_Latn.yaml' 2026-05-11T14:30:07,325 adding 'lm_eval/tasks/belebele/belebele_ceb_Latn.yaml' 2026-05-11T14:30:07,326 adding 'lm_eval/tasks/belebele/belebele_ces_Latn.yaml' 2026-05-11T14:30:07,327 adding 'lm_eval/tasks/belebele/belebele_ckb_Arab.yaml' 2026-05-11T14:30:07,328 adding 'lm_eval/tasks/belebele/belebele_dan_Latn.yaml' 2026-05-11T14:30:07,329 adding 'lm_eval/tasks/belebele/belebele_deu_Latn.yaml' 2026-05-11T14:30:07,330 adding 'lm_eval/tasks/belebele/belebele_ell_Grek.yaml' 2026-05-11T14:30:07,331 adding 'lm_eval/tasks/belebele/belebele_eng_Latn.yaml' 2026-05-11T14:30:07,333 adding 'lm_eval/tasks/belebele/belebele_est_Latn.yaml' 2026-05-11T14:30:07,334 adding 'lm_eval/tasks/belebele/belebele_eus_Latn.yaml' 2026-05-11T14:30:07,335 adding 'lm_eval/tasks/belebele/belebele_fin_Latn.yaml' 2026-05-11T14:30:07,336 adding 'lm_eval/tasks/belebele/belebele_fra_Latn.yaml' 2026-05-11T14:30:07,337 adding 'lm_eval/tasks/belebele/belebele_fuv_Latn.yaml' 2026-05-11T14:30:07,339 adding 'lm_eval/tasks/belebele/belebele_gaz_Latn.yaml' 2026-05-11T14:30:07,340 adding 'lm_eval/tasks/belebele/belebele_grn_Latn.yaml' 2026-05-11T14:30:07,341 adding 'lm_eval/tasks/belebele/belebele_guj_Gujr.yaml' 2026-05-11T14:30:07,342 adding 'lm_eval/tasks/belebele/belebele_hat_Latn.yaml' 2026-05-11T14:30:07,343 adding 'lm_eval/tasks/belebele/belebele_hau_Latn.yaml' 2026-05-11T14:30:07,344 adding 'lm_eval/tasks/belebele/belebele_heb_Hebr.yaml' 2026-05-11T14:30:07,345 adding 'lm_eval/tasks/belebele/belebele_hin_Deva.yaml' 2026-05-11T14:30:07,346 adding 'lm_eval/tasks/belebele/belebele_hin_Latn.yaml' 2026-05-11T14:30:07,347 adding 'lm_eval/tasks/belebele/belebele_hrv_Latn.yaml' 2026-05-11T14:30:07,348 adding 'lm_eval/tasks/belebele/belebele_hun_Latn.yaml' 2026-05-11T14:30:07,350 adding 'lm_eval/tasks/belebele/belebele_hye_Armn.yaml' 2026-05-11T14:30:07,351 adding 'lm_eval/tasks/belebele/belebele_ibo_Latn.yaml' 2026-05-11T14:30:07,352 adding 'lm_eval/tasks/belebele/belebele_ilo_Latn.yaml' 2026-05-11T14:30:07,353 adding 'lm_eval/tasks/belebele/belebele_ind_Latn.yaml' 2026-05-11T14:30:07,354 adding 'lm_eval/tasks/belebele/belebele_isl_Latn.yaml' 2026-05-11T14:30:07,355 adding 'lm_eval/tasks/belebele/belebele_ita_Latn.yaml' 2026-05-11T14:30:07,357 adding 'lm_eval/tasks/belebele/belebele_jav_Latn.yaml' 2026-05-11T14:30:07,358 adding 'lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml' 2026-05-11T14:30:07,359 adding 'lm_eval/tasks/belebele/belebele_kac_Latn.yaml' 2026-05-11T14:30:07,360 adding 'lm_eval/tasks/belebele/belebele_kan_Knda.yaml' 2026-05-11T14:30:07,361 adding 'lm_eval/tasks/belebele/belebele_kat_Geor.yaml' 2026-05-11T14:30:07,362 adding 'lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml' 2026-05-11T14:30:07,363 adding 'lm_eval/tasks/belebele/belebele_kea_Latn.yaml' 2026-05-11T14:30:07,365 adding 'lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml' 2026-05-11T14:30:07,366 adding 'lm_eval/tasks/belebele/belebele_khm_Khmr.yaml' 2026-05-11T14:30:07,367 adding 'lm_eval/tasks/belebele/belebele_kin_Latn.yaml' 2026-05-11T14:30:07,368 adding 'lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml' 2026-05-11T14:30:07,369 adding 'lm_eval/tasks/belebele/belebele_kor_Hang.yaml' 2026-05-11T14:30:07,370 adding 'lm_eval/tasks/belebele/belebele_lao_Laoo.yaml' 2026-05-11T14:30:07,371 adding 'lm_eval/tasks/belebele/belebele_lin_Latn.yaml' 2026-05-11T14:30:07,372 adding 'lm_eval/tasks/belebele/belebele_lit_Latn.yaml' 2026-05-11T14:30:07,373 adding 'lm_eval/tasks/belebele/belebele_lug_Latn.yaml' 2026-05-11T14:30:07,375 adding 'lm_eval/tasks/belebele/belebele_luo_Latn.yaml' 2026-05-11T14:30:07,376 adding 'lm_eval/tasks/belebele/belebele_lvs_Latn.yaml' 2026-05-11T14:30:07,377 adding 'lm_eval/tasks/belebele/belebele_mal_Mlym.yaml' 2026-05-11T14:30:07,378 adding 'lm_eval/tasks/belebele/belebele_mar_Deva.yaml' 2026-05-11T14:30:07,379 adding 'lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml' 2026-05-11T14:30:07,380 adding 'lm_eval/tasks/belebele/belebele_mlt_Latn.yaml' 2026-05-11T14:30:07,381 adding 'lm_eval/tasks/belebele/belebele_mri_Latn.yaml' 2026-05-11T14:30:07,383 adding 'lm_eval/tasks/belebele/belebele_mya_Mymr.yaml' 2026-05-11T14:30:07,384 adding 'lm_eval/tasks/belebele/belebele_nld_Latn.yaml' 2026-05-11T14:30:07,385 adding 'lm_eval/tasks/belebele/belebele_nob_Latn.yaml' 2026-05-11T14:30:07,386 adding 'lm_eval/tasks/belebele/belebele_npi_Deva.yaml' 2026-05-11T14:30:07,387 adding 'lm_eval/tasks/belebele/belebele_npi_Latn.yaml' 2026-05-11T14:30:07,389 adding 'lm_eval/tasks/belebele/belebele_nso_Latn.yaml' 2026-05-11T14:30:07,390 adding 'lm_eval/tasks/belebele/belebele_nya_Latn.yaml' 2026-05-11T14:30:07,392 adding 'lm_eval/tasks/belebele/belebele_ory_Orya.yaml' 2026-05-11T14:30:07,393 adding 'lm_eval/tasks/belebele/belebele_pan_Guru.yaml' 2026-05-11T14:30:07,394 adding 'lm_eval/tasks/belebele/belebele_pbt_Arab.yaml' 2026-05-11T14:30:07,395 adding 'lm_eval/tasks/belebele/belebele_pes_Arab.yaml' 2026-05-11T14:30:07,396 adding 'lm_eval/tasks/belebele/belebele_plt_Latn.yaml' 2026-05-11T14:30:07,397 adding 'lm_eval/tasks/belebele/belebele_pol_Latn.yaml' 2026-05-11T14:30:07,398 adding 'lm_eval/tasks/belebele/belebele_por_Latn.yaml' 2026-05-11T14:30:07,400 adding 'lm_eval/tasks/belebele/belebele_ron_Latn.yaml' 2026-05-11T14:30:07,401 adding 'lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml' 2026-05-11T14:30:07,402 adding 'lm_eval/tasks/belebele/belebele_shn_Mymr.yaml' 2026-05-11T14:30:07,403 adding 'lm_eval/tasks/belebele/belebele_sin_Latn.yaml' 2026-05-11T14:30:07,404 adding 'lm_eval/tasks/belebele/belebele_sin_Sinh.yaml' 2026-05-11T14:30:07,405 adding 'lm_eval/tasks/belebele/belebele_slk_Latn.yaml' 2026-05-11T14:30:07,406 adding 'lm_eval/tasks/belebele/belebele_slv_Latn.yaml' 2026-05-11T14:30:07,407 adding 'lm_eval/tasks/belebele/belebele_sna_Latn.yaml' 2026-05-11T14:30:07,408 adding 'lm_eval/tasks/belebele/belebele_snd_Arab.yaml' 2026-05-11T14:30:07,409 adding 'lm_eval/tasks/belebele/belebele_som_Latn.yaml' 2026-05-11T14:30:07,410 adding 'lm_eval/tasks/belebele/belebele_sot_Latn.yaml' 2026-05-11T14:30:07,411 adding 'lm_eval/tasks/belebele/belebele_spa_Latn.yaml' 2026-05-11T14:30:07,412 adding 'lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml' 2026-05-11T14:30:07,413 adding 'lm_eval/tasks/belebele/belebele_ssw_Latn.yaml' 2026-05-11T14:30:07,414 adding 'lm_eval/tasks/belebele/belebele_sun_Latn.yaml' 2026-05-11T14:30:07,415 adding 'lm_eval/tasks/belebele/belebele_swe_Latn.yaml' 2026-05-11T14:30:07,416 adding 'lm_eval/tasks/belebele/belebele_swh_Latn.yaml' 2026-05-11T14:30:07,417 adding 'lm_eval/tasks/belebele/belebele_tam_Taml.yaml' 2026-05-11T14:30:07,419 adding 'lm_eval/tasks/belebele/belebele_tel_Telu.yaml' 2026-05-11T14:30:07,420 adding 'lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml' 2026-05-11T14:30:07,421 adding 'lm_eval/tasks/belebele/belebele_tgl_Latn.yaml' 2026-05-11T14:30:07,422 adding 'lm_eval/tasks/belebele/belebele_tha_Thai.yaml' 2026-05-11T14:30:07,423 adding 'lm_eval/tasks/belebele/belebele_tir_Ethi.yaml' 2026-05-11T14:30:07,424 adding 'lm_eval/tasks/belebele/belebele_tsn_Latn.yaml' 2026-05-11T14:30:07,425 adding 'lm_eval/tasks/belebele/belebele_tso_Latn.yaml' 2026-05-11T14:30:07,426 adding 'lm_eval/tasks/belebele/belebele_tur_Latn.yaml' 2026-05-11T14:30:07,427 adding 'lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml' 2026-05-11T14:30:07,429 adding 'lm_eval/tasks/belebele/belebele_urd_Arab.yaml' 2026-05-11T14:30:07,430 adding 'lm_eval/tasks/belebele/belebele_urd_Latn.yaml' 2026-05-11T14:30:07,431 adding 'lm_eval/tasks/belebele/belebele_uzn_Latn.yaml' 2026-05-11T14:30:07,432 adding 'lm_eval/tasks/belebele/belebele_vie_Latn.yaml' 2026-05-11T14:30:07,433 adding 'lm_eval/tasks/belebele/belebele_war_Latn.yaml' 2026-05-11T14:30:07,434 adding 'lm_eval/tasks/belebele/belebele_wol_Latn.yaml' 2026-05-11T14:30:07,435 adding 'lm_eval/tasks/belebele/belebele_xho_Latn.yaml' 2026-05-11T14:30:07,437 adding 'lm_eval/tasks/belebele/belebele_yor_Latn.yaml' 2026-05-11T14:30:07,438 adding 'lm_eval/tasks/belebele/belebele_zho_Hans.yaml' 2026-05-11T14:30:07,439 adding 'lm_eval/tasks/belebele/belebele_zho_Hant.yaml' 2026-05-11T14:30:07,440 adding 'lm_eval/tasks/belebele/belebele_zsm_Latn.yaml' 2026-05-11T14:30:07,441 adding 'lm_eval/tasks/belebele/belebele_zul_Latn.yaml' 2026-05-11T14:30:07,443 adding 'lm_eval/tasks/benchmarks/README.md' 2026-05-11T14:30:07,444 adding 'lm_eval/tasks/benchmarks/minerva_math.yaml' 2026-05-11T14:30:07,445 adding 'lm_eval/tasks/benchmarks/openllm.yaml' 2026-05-11T14:30:07,446 adding 'lm_eval/tasks/benchmarks/pythia.yaml' 2026-05-11T14:30:07,448 adding 'lm_eval/tasks/benchmarks/t0_eval.yaml' 2026-05-11T14:30:07,450 adding 'lm_eval/tasks/benchmarks/flan/_held_in_template_yaml' 2026-05-11T14:30:07,451 adding 'lm_eval/tasks/benchmarks/flan/flan_held_in.yaml' 2026-05-11T14:30:07,453 adding 'lm_eval/tasks/benchmarks/flan/flan_held_out.yaml' 2026-05-11T14:30:07,455 adding 'lm_eval/tasks/benchmarks/multimedqa/README.md' 2026-05-11T14:30:07,456 adding 'lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml' 2026-05-11T14:30:07,458 adding 'lm_eval/tasks/bertaqa/README.md' 2026-05-11T14:30:07,459 adding 'lm_eval/tasks/bertaqa/_bertaqa_template' 2026-05-11T14:30:07,460 adding 'lm_eval/tasks/bertaqa/bertaqa_en.yaml' 2026-05-11T14:30:07,461 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml' 2026-05-11T14:30:07,462 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml' 2026-05-11T14:30:07,463 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml' 2026-05-11T14:30:07,464 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml' 2026-05-11T14:30:07,465 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml' 2026-05-11T14:30:07,467 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml' 2026-05-11T14:30:07,468 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml' 2026-05-11T14:30:07,469 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml' 2026-05-11T14:30:07,470 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml' 2026-05-11T14:30:07,471 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml' 2026-05-11T14:30:07,472 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml' 2026-05-11T14:30:07,473 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml' 2026-05-11T14:30:07,474 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml' 2026-05-11T14:30:07,475 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml' 2026-05-11T14:30:07,476 adding 'lm_eval/tasks/bertaqa/bertaqa_eu.yaml' 2026-05-11T14:30:07,479 adding 'lm_eval/tasks/bhs/README.md' 2026-05-11T14:30:07,480 adding 'lm_eval/tasks/bhs/_template_yaml' 2026-05-11T14:30:07,481 adding 'lm_eval/tasks/bhs/basque-DO-S_DO_V_AUX.yaml' 2026-05-11T14:30:07,482 adding 'lm_eval/tasks/bhs/basque-DO-S_IO_DO_V_AUX.yaml' 2026-05-11T14:30:07,483 adding 'lm_eval/tasks/bhs/basque-IO-IO_S_V_AUX.yaml' 2026-05-11T14:30:07,484 adding 'lm_eval/tasks/bhs/basque-IO-S_IO_DO_V_AUX.yaml' 2026-05-11T14:30:07,485 adding 'lm_eval/tasks/bhs/basque-S-IO_S_V_AUX.yaml' 2026-05-11T14:30:07,486 adding 'lm_eval/tasks/bhs/basque-S-S_DO_V_AUX.yaml' 2026-05-11T14:30:07,488 adding 'lm_eval/tasks/bhs/basque-S-S_IO_DO_V_AUX.yaml' 2026-05-11T14:30:07,489 adding 'lm_eval/tasks/bhs/basque-S-S_V_AUX.yaml' 2026-05-11T14:30:07,490 adding 'lm_eval/tasks/bhs/bhs_basque.yaml' 2026-05-11T14:30:07,491 adding 'lm_eval/tasks/bhs/bhs_hindi.yaml' 2026-05-11T14:30:07,492 adding 'lm_eval/tasks/bhs/bhs_swahili.yaml' 2026-05-11T14:30:07,493 adding 'lm_eval/tasks/bhs/hindi-S_O_V.yaml' 2026-05-11T14:30:07,494 adding 'lm_eval/tasks/bhs/hindi-S_PossPRN_O_V.yaml' 2026-05-11T14:30:07,495 adding 'lm_eval/tasks/bhs/hindi-S_PossPRN_PossN_O_V.yaml' 2026-05-11T14:30:07,497 adding 'lm_eval/tasks/bhs/hindi-S_ne_O_V.yaml' 2026-05-11T14:30:07,498 adding 'lm_eval/tasks/bhs/hindi-S_ne_PossPRN_O_V.yaml' 2026-05-11T14:30:07,499 adding 'lm_eval/tasks/bhs/hindi-S_ne_PossPRN_PossN_O_V.yaml' 2026-05-11T14:30:07,500 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_V_ni_AN.yaml' 2026-05-11T14:30:07,501 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_ni_AN.yaml' 2026-05-11T14:30:07,502 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V.yaml' 2026-05-11T14:30:07,503 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V1_V2.yaml' 2026-05-11T14:30:07,505 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_V.yaml' 2026-05-11T14:30:07,506 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_ni_A.yaml' 2026-05-11T14:30:07,507 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_V.yaml' 2026-05-11T14:30:07,508 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_ni_A.yaml' 2026-05-11T14:30:07,511 adding 'lm_eval/tasks/bigbench/README.md' 2026-05-11T14:30:07,512 adding 'lm_eval/tasks/bigbench/generate_tasks.py' 2026-05-11T14:30:07,514 adding 'lm_eval/tasks/bigbench/generate_until_template_yaml' 2026-05-11T14:30:07,515 adding 'lm_eval/tasks/bigbench/multiple_choice_template_a_yaml' 2026-05-11T14:30:07,516 adding 'lm_eval/tasks/bigbench/multiple_choice_template_b_yaml' 2026-05-11T14:30:07,517 adding 'lm_eval/tasks/bigbench/push_bigbench_dataset.py' 2026-05-11T14:30:07,518 adding 'lm_eval/tasks/bigbench/utils.py' 2026-05-11T14:30:07,523 adding 'lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml' 2026-05-11T14:30:07,524 adding 'lm_eval/tasks/bigbench/generate_until/anachronisms.yaml' 2026-05-11T14:30:07,525 adding 'lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml' 2026-05-11T14:30:07,527 adding 'lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml' 2026-05-11T14:30:07,528 adding 'lm_eval/tasks/bigbench/generate_until/arithmetic.yaml' 2026-05-11T14:30:07,529 adding 'lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml' 2026-05-11T14:30:07,530 adding 'lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml' 2026-05-11T14:30:07,531 adding 'lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml' 2026-05-11T14:30:07,532 adding 'lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml' 2026-05-11T14:30:07,533 adding 'lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml' 2026-05-11T14:30:07,534 adding 'lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml' 2026-05-11T14:30:07,535 adding 'lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml' 2026-05-11T14:30:07,536 adding 'lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml' 2026-05-11T14:30:07,537 adding 'lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml' 2026-05-11T14:30:07,539 adding 'lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml' 2026-05-11T14:30:07,540 adding 'lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml' 2026-05-11T14:30:07,541 adding 'lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml' 2026-05-11T14:30:07,542 adding 'lm_eval/tasks/bigbench/generate_until/code_line_description.yaml' 2026-05-11T14:30:07,543 adding 'lm_eval/tasks/bigbench/generate_until/codenames.yaml' 2026-05-11T14:30:07,544 adding 'lm_eval/tasks/bigbench/generate_until/color.yaml' 2026-05-11T14:30:07,545 adding 'lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml' 2026-05-11T14:30:07,546 adding 'lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml' 2026-05-11T14:30:07,548 adding 'lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml' 2026-05-11T14:30:07,549 adding 'lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml' 2026-05-11T14:30:07,550 adding 'lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml' 2026-05-11T14:30:07,551 adding 'lm_eval/tasks/bigbench/generate_until/crass_ai.yaml' 2026-05-11T14:30:07,552 adding 'lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml' 2026-05-11T14:30:07,553 adding 'lm_eval/tasks/bigbench/generate_until/cryptonite.yaml' 2026-05-11T14:30:07,554 adding 'lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml' 2026-05-11T14:30:07,556 adding 'lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml' 2026-05-11T14:30:07,557 adding 'lm_eval/tasks/bigbench/generate_until/date_understanding.yaml' 2026-05-11T14:30:07,558 adding 'lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml' 2026-05-11T14:30:07,559 adding 'lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml' 2026-05-11T14:30:07,560 adding 'lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml' 2026-05-11T14:30:07,561 adding 'lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml' 2026-05-11T14:30:07,563 adding 'lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml' 2026-05-11T14:30:07,564 adding 'lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml' 2026-05-11T14:30:07,565 adding 'lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml' 2026-05-11T14:30:07,566 adding 'lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml' 2026-05-11T14:30:07,567 adding 'lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml' 2026-05-11T14:30:07,568 adding 'lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml' 2026-05-11T14:30:07,569 adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml' 2026-05-11T14:30:07,571 adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml' 2026-05-11T14:30:07,572 adding 'lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml' 2026-05-11T14:30:07,573 adding 'lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml' 2026-05-11T14:30:07,574 adding 'lm_eval/tasks/bigbench/generate_until/fact_checker.yaml' 2026-05-11T14:30:07,575 adding 'lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml' 2026-05-11T14:30:07,576 adding 'lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml' 2026-05-11T14:30:07,577 adding 'lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml' 2026-05-11T14:30:07,578 adding 'lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml' 2026-05-11T14:30:07,579 adding 'lm_eval/tasks/bigbench/generate_until/gem.yaml' 2026-05-11T14:30:07,580 adding 'lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml' 2026-05-11T14:30:07,581 adding 'lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml' 2026-05-11T14:30:07,582 adding 'lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml' 2026-05-11T14:30:07,583 adding 'lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml' 2026-05-11T14:30:07,585 adding 'lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml' 2026-05-11T14:30:07,586 adding 'lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml' 2026-05-11T14:30:07,587 adding 'lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml' 2026-05-11T14:30:07,588 adding 'lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml' 2026-05-11T14:30:07,589 adding 'lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml' 2026-05-11T14:30:07,590 adding 'lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml' 2026-05-11T14:30:07,591 adding 'lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml' 2026-05-11T14:30:07,592 adding 'lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml' 2026-05-11T14:30:07,593 adding 'lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml' 2026-05-11T14:30:07,594 adding 'lm_eval/tasks/bigbench/generate_until/implicatures.yaml' 2026-05-11T14:30:07,596 adding 'lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml' 2026-05-11T14:30:07,597 adding 'lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml' 2026-05-11T14:30:07,598 adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml' 2026-05-11T14:30:07,599 adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml' 2026-05-11T14:30:07,600 adding 'lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml' 2026-05-11T14:30:07,601 adding 'lm_eval/tasks/bigbench/generate_until/irony_identification.yaml' 2026-05-11T14:30:07,603 adding 'lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml' 2026-05-11T14:30:07,604 adding 'lm_eval/tasks/bigbench/generate_until/kannada.yaml' 2026-05-11T14:30:07,605 adding 'lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml' 2026-05-11T14:30:07,606 adding 'lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml' 2026-05-11T14:30:07,607 adding 'lm_eval/tasks/bigbench/generate_until/language_games.yaml' 2026-05-11T14:30:07,608 adding 'lm_eval/tasks/bigbench/generate_until/language_identification.yaml' 2026-05-11T14:30:07,610 adding 'lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml' 2026-05-11T14:30:07,611 adding 'lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml' 2026-05-11T14:30:07,612 adding 'lm_eval/tasks/bigbench/generate_until/list_functions.yaml' 2026-05-11T14:30:07,613 adding 'lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml' 2026-05-11T14:30:07,614 adding 'lm_eval/tasks/bigbench/generate_until/logical_args.yaml' 2026-05-11T14:30:07,615 adding 'lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml' 2026-05-11T14:30:07,617 adding 'lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml' 2026-05-11T14:30:07,618 adding 'lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml' 2026-05-11T14:30:07,619 adding 'lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml' 2026-05-11T14:30:07,620 adding 'lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml' 2026-05-11T14:30:07,622 adding 'lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml' 2026-05-11T14:30:07,623 adding 'lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml' 2026-05-11T14:30:07,624 adding 'lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml' 2026-05-11T14:30:07,625 adding 'lm_eval/tasks/bigbench/generate_until/misconceptions.yaml' 2026-05-11T14:30:07,626 adding 'lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml' 2026-05-11T14:30:07,627 adding 'lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml' 2026-05-11T14:30:07,628 adding 'lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml' 2026-05-11T14:30:07,629 adding 'lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml' 2026-05-11T14:30:07,630 adding 'lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml' 2026-05-11T14:30:07,632 adding 'lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml' 2026-05-11T14:30:07,633 adding 'lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml' 2026-05-11T14:30:07,634 adding 'lm_eval/tasks/bigbench/generate_until/multiemo.yaml' 2026-05-11T14:30:07,635 adding 'lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml' 2026-05-11T14:30:07,636 adding 'lm_eval/tasks/bigbench/generate_until/navigate.yaml' 2026-05-11T14:30:07,637 adding 'lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml' 2026-05-11T14:30:07,638 adding 'lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml' 2026-05-11T14:30:07,639 adding 'lm_eval/tasks/bigbench/generate_until/object_counting.yaml' 2026-05-11T14:30:07,641 adding 'lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml' 2026-05-11T14:30:07,642 adding 'lm_eval/tasks/bigbench/generate_until/operators.yaml' 2026-05-11T14:30:07,643 adding 'lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml' 2026-05-11T14:30:07,644 adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml' 2026-05-11T14:30:07,645 adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml' 2026-05-11T14:30:07,646 adding 'lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml' 2026-05-11T14:30:07,647 adding 'lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml' 2026-05-11T14:30:07,648 adding 'lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml' 2026-05-11T14:30:07,650 adding 'lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml' 2026-05-11T14:30:07,651 adding 'lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml' 2026-05-11T14:30:07,652 adding 'lm_eval/tasks/bigbench/generate_until/physics.yaml' 2026-05-11T14:30:07,653 adding 'lm_eval/tasks/bigbench/generate_until/physics_questions.yaml' 2026-05-11T14:30:07,654 adding 'lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml' 2026-05-11T14:30:07,655 adding 'lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml' 2026-05-11T14:30:07,657 adding 'lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml' 2026-05-11T14:30:07,658 adding 'lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml' 2026-05-11T14:30:07,659 adding 'lm_eval/tasks/bigbench/generate_until/question_selection.yaml' 2026-05-11T14:30:07,660 adding 'lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml' 2026-05-11T14:30:07,661 adding 'lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml' 2026-05-11T14:30:07,662 adding 'lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml' 2026-05-11T14:30:07,663 adding 'lm_eval/tasks/bigbench/generate_until/rephrase.yaml' 2026-05-11T14:30:07,665 adding 'lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml' 2026-05-11T14:30:07,666 adding 'lm_eval/tasks/bigbench/generate_until/ruin_names.yaml' 2026-05-11T14:30:07,667 adding 'lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml' 2026-05-11T14:30:07,668 adding 'lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml' 2026-05-11T14:30:07,669 adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml' 2026-05-11T14:30:07,670 adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml' 2026-05-11T14:30:07,671 adding 'lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml' 2026-05-11T14:30:07,672 adding 'lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml' 2026-05-11T14:30:07,673 adding 'lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml' 2026-05-11T14:30:07,674 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml' 2026-05-11T14:30:07,675 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml' 2026-05-11T14:30:07,676 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml' 2026-05-11T14:30:07,677 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml' 2026-05-11T14:30:07,679 adding 'lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml' 2026-05-11T14:30:07,680 adding 'lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml' 2026-05-11T14:30:07,681 adding 'lm_eval/tasks/bigbench/generate_until/snarks.yaml' 2026-05-11T14:30:07,682 adding 'lm_eval/tasks/bigbench/generate_until/social_iqa.yaml' 2026-05-11T14:30:07,683 adding 'lm_eval/tasks/bigbench/generate_until/social_support.yaml' 2026-05-11T14:30:07,684 adding 'lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml' 2026-05-11T14:30:07,685 adding 'lm_eval/tasks/bigbench/generate_until/strange_stories.yaml' 2026-05-11T14:30:07,686 adding 'lm_eval/tasks/bigbench/generate_until/strategyqa.yaml' 2026-05-11T14:30:07,687 adding 'lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml' 2026-05-11T14:30:07,688 adding 'lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml' 2026-05-11T14:30:07,690 adding 'lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml' 2026-05-11T14:30:07,691 adding 'lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml' 2026-05-11T14:30:07,692 adding 'lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml' 2026-05-11T14:30:07,693 adding 'lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml' 2026-05-11T14:30:07,694 adding 'lm_eval/tasks/bigbench/generate_until/tense.yaml' 2026-05-11T14:30:07,695 adding 'lm_eval/tasks/bigbench/generate_until/timedial.yaml' 2026-05-11T14:30:07,696 adding 'lm_eval/tasks/bigbench/generate_until/topical_chat.yaml' 2026-05-11T14:30:07,698 adding 'lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml' 2026-05-11T14:30:07,699 adding 'lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml' 2026-05-11T14:30:07,700 adding 'lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml' 2026-05-11T14:30:07,701 adding 'lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml' 2026-05-11T14:30:07,702 adding 'lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml' 2026-05-11T14:30:07,703 adding 'lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml' 2026-05-11T14:30:07,704 adding 'lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml' 2026-05-11T14:30:07,706 adding 'lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml' 2026-05-11T14:30:07,707 adding 'lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml' 2026-05-11T14:30:07,708 adding 'lm_eval/tasks/bigbench/generate_until/winowhy.yaml' 2026-05-11T14:30:07,709 adding 'lm_eval/tasks/bigbench/generate_until/word_sorting.yaml' 2026-05-11T14:30:07,710 adding 'lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml' 2026-05-11T14:30:07,714 adding 'lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml' 2026-05-11T14:30:07,715 adding 'lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml' 2026-05-11T14:30:07,716 adding 'lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml' 2026-05-11T14:30:07,717 adding 'lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml' 2026-05-11T14:30:07,718 adding 'lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml' 2026-05-11T14:30:07,719 adding 'lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml' 2026-05-11T14:30:07,720 adding 'lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml' 2026-05-11T14:30:07,722 adding 'lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml' 2026-05-11T14:30:07,723 adding 'lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml' 2026-05-11T14:30:07,724 adding 'lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml' 2026-05-11T14:30:07,725 adding 'lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml' 2026-05-11T14:30:07,726 adding 'lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml' 2026-05-11T14:30:07,727 adding 'lm_eval/tasks/bigbench/multiple_choice/color.yaml' 2026-05-11T14:30:07,728 adding 'lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml' 2026-05-11T14:30:07,729 adding 'lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml' 2026-05-11T14:30:07,730 adding 'lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml' 2026-05-11T14:30:07,731 adding 'lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml' 2026-05-11T14:30:07,732 adding 'lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml' 2026-05-11T14:30:07,733 adding 'lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml' 2026-05-11T14:30:07,735 adding 'lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml' 2026-05-11T14:30:07,736 adding 'lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml' 2026-05-11T14:30:07,737 adding 'lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml' 2026-05-11T14:30:07,738 adding 'lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml' 2026-05-11T14:30:07,739 adding 'lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml' 2026-05-11T14:30:07,740 adding 'lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml' 2026-05-11T14:30:07,741 adding 'lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml' 2026-05-11T14:30:07,742 adding 'lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml' 2026-05-11T14:30:07,744 adding 'lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml' 2026-05-11T14:30:07,745 adding 'lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml' 2026-05-11T14:30:07,746 adding 'lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml' 2026-05-11T14:30:07,747 adding 'lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml' 2026-05-11T14:30:07,748 adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml' 2026-05-11T14:30:07,749 adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml' 2026-05-11T14:30:07,751 adding 'lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml' 2026-05-11T14:30:07,752 adding 'lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml' 2026-05-11T14:30:07,753 adding 'lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml' 2026-05-11T14:30:07,754 adding 'lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml' 2026-05-11T14:30:07,755 adding 'lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml' 2026-05-11T14:30:07,757 adding 'lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml' 2026-05-11T14:30:07,758 adding 'lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml' 2026-05-11T14:30:07,759 adding 'lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml' 2026-05-11T14:30:07,760 adding 'lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml' 2026-05-11T14:30:07,761 adding 'lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml' 2026-05-11T14:30:07,762 adding 'lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml' 2026-05-11T14:30:07,764 adding 'lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml' 2026-05-11T14:30:07,765 adding 'lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml' 2026-05-11T14:30:07,766 adding 'lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml' 2026-05-11T14:30:07,767 adding 'lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml' 2026-05-11T14:30:07,768 adding 'lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml' 2026-05-11T14:30:07,769 adding 'lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml' 2026-05-11T14:30:07,770 adding 'lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml' 2026-05-11T14:30:07,771 adding 'lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml' 2026-05-11T14:30:07,772 adding 'lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml' 2026-05-11T14:30:07,773 adding 'lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml' 2026-05-11T14:30:07,774 adding 'lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml' 2026-05-11T14:30:07,775 adding 'lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml' 2026-05-11T14:30:07,776 adding 'lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml' 2026-05-11T14:30:07,778 adding 'lm_eval/tasks/bigbench/multiple_choice/kannada.yaml' 2026-05-11T14:30:07,779 adding 'lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml' 2026-05-11T14:30:07,780 adding 'lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml' 2026-05-11T14:30:07,781 adding 'lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml' 2026-05-11T14:30:07,782 adding 'lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml' 2026-05-11T14:30:07,783 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml' 2026-05-11T14:30:07,784 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml' 2026-05-11T14:30:07,785 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml' 2026-05-11T14:30:07,787 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml' 2026-05-11T14:30:07,788 adding 'lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml' 2026-05-11T14:30:07,789 adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml' 2026-05-11T14:30:07,790 adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml' 2026-05-11T14:30:07,791 adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml' 2026-05-11T14:30:07,792 adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml' 2026-05-11T14:30:07,793 adding 'lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml' 2026-05-11T14:30:07,794 adding 'lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml' 2026-05-11T14:30:07,796 adding 'lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml' 2026-05-11T14:30:07,797 adding 'lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml' 2026-05-11T14:30:07,798 adding 'lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml' 2026-05-11T14:30:07,799 adding 'lm_eval/tasks/bigbench/multiple_choice/navigate.yaml' 2026-05-11T14:30:07,800 adding 'lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml' 2026-05-11T14:30:07,801 adding 'lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml' 2026-05-11T14:30:07,803 adding 'lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml' 2026-05-11T14:30:07,804 adding 'lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml' 2026-05-11T14:30:07,805 adding 'lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml' 2026-05-11T14:30:07,806 adding 'lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml' 2026-05-11T14:30:07,807 adding 'lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml' 2026-05-11T14:30:07,808 adding 'lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml' 2026-05-11T14:30:07,809 adding 'lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml' 2026-05-11T14:30:07,811 adding 'lm_eval/tasks/bigbench/multiple_choice/physics.yaml' 2026-05-11T14:30:07,812 adding 'lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml' 2026-05-11T14:30:07,813 adding 'lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml' 2026-05-11T14:30:07,814 adding 'lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml' 2026-05-11T14:30:07,815 adding 'lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml' 2026-05-11T14:30:07,816 adding 'lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml' 2026-05-11T14:30:07,817 adding 'lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml' 2026-05-11T14:30:07,818 adding 'lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml' 2026-05-11T14:30:07,819 adding 'lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml' 2026-05-11T14:30:07,820 adding 'lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml' 2026-05-11T14:30:07,821 adding 'lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml' 2026-05-11T14:30:07,822 adding 'lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml' 2026-05-11T14:30:07,824 adding 'lm_eval/tasks/bigbench/multiple_choice/snarks.yaml' 2026-05-11T14:30:07,825 adding 'lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml' 2026-05-11T14:30:07,826 adding 'lm_eval/tasks/bigbench/multiple_choice/social_support.yaml' 2026-05-11T14:30:07,827 adding 'lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml' 2026-05-11T14:30:07,828 adding 'lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml' 2026-05-11T14:30:07,829 adding 'lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml' 2026-05-11T14:30:07,830 adding 'lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml' 2026-05-11T14:30:07,831 adding 'lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml' 2026-05-11T14:30:07,832 adding 'lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml' 2026-05-11T14:30:07,833 adding 'lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml' 2026-05-11T14:30:07,835 adding 'lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml' 2026-05-11T14:30:07,836 adding 'lm_eval/tasks/bigbench/multiple_choice/timedial.yaml' 2026-05-11T14:30:07,837 adding 'lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml' 2026-05-11T14:30:07,838 adding 'lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml' 2026-05-11T14:30:07,839 adding 'lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml' 2026-05-11T14:30:07,840 adding 'lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml' 2026-05-11T14:30:07,841 adding 'lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml' 2026-05-11T14:30:07,843 adding 'lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml' 2026-05-11T14:30:07,844 adding 'lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml' 2026-05-11T14:30:07,845 adding 'lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml' 2026-05-11T14:30:07,846 adding 'lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml' 2026-05-11T14:30:07,849 adding 'lm_eval/tasks/blimp/README.md' 2026-05-11T14:30:07,850 adding 'lm_eval/tasks/blimp/_blimp.yaml' 2026-05-11T14:30:07,852 adding 'lm_eval/tasks/blimp/_template_yaml' 2026-05-11T14:30:07,853 adding 'lm_eval/tasks/blimp/adjunct_island.yaml' 2026-05-11T14:30:07,854 adding 'lm_eval/tasks/blimp/anaphor_gender_agreement.yaml' 2026-05-11T14:30:07,855 adding 'lm_eval/tasks/blimp/anaphor_number_agreement.yaml' 2026-05-11T14:30:07,856 adding 'lm_eval/tasks/blimp/animate_subject_passive.yaml' 2026-05-11T14:30:07,857 adding 'lm_eval/tasks/blimp/animate_subject_trans.yaml' 2026-05-11T14:30:07,859 adding 'lm_eval/tasks/blimp/causative.yaml' 2026-05-11T14:30:07,860 adding 'lm_eval/tasks/blimp/complex_NP_island.yaml' 2026-05-11T14:30:07,861 adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml' 2026-05-11T14:30:07,862 adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml' 2026-05-11T14:30:07,863 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml' 2026-05-11T14:30:07,864 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml' 2026-05-11T14:30:07,865 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml' 2026-05-11T14:30:07,866 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml' 2026-05-11T14:30:07,867 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml' 2026-05-11T14:30:07,869 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml' 2026-05-11T14:30:07,870 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml' 2026-05-11T14:30:07,871 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml' 2026-05-11T14:30:07,872 adding 'lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml' 2026-05-11T14:30:07,873 adding 'lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml' 2026-05-11T14:30:07,874 adding 'lm_eval/tasks/blimp/drop_argument.yaml' 2026-05-11T14:30:07,876 adding 'lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml' 2026-05-11T14:30:07,877 adding 'lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml' 2026-05-11T14:30:07,878 adding 'lm_eval/tasks/blimp/existential_there_object_raising.yaml' 2026-05-11T14:30:07,879 adding 'lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml' 2026-05-11T14:30:07,880 adding 'lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml' 2026-05-11T14:30:07,881 adding 'lm_eval/tasks/blimp/existential_there_subject_raising.yaml' 2026-05-11T14:30:07,882 adding 'lm_eval/tasks/blimp/expletive_it_object_raising.yaml' 2026-05-11T14:30:07,884 adding 'lm_eval/tasks/blimp/generate_configs.py' 2026-05-11T14:30:07,885 adding 'lm_eval/tasks/blimp/inchoative.yaml' 2026-05-11T14:30:07,886 adding 'lm_eval/tasks/blimp/intransitive.yaml' 2026-05-11T14:30:07,887 adding 'lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml' 2026-05-11T14:30:07,888 adding 'lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml' 2026-05-11T14:30:07,889 adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml' 2026-05-11T14:30:07,891 adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml' 2026-05-11T14:30:07,892 adding 'lm_eval/tasks/blimp/left_branch_island_echo_question.yaml' 2026-05-11T14:30:07,893 adding 'lm_eval/tasks/blimp/left_branch_island_simple_question.yaml' 2026-05-11T14:30:07,894 adding 'lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml' 2026-05-11T14:30:07,895 adding 'lm_eval/tasks/blimp/npi_present_1.yaml' 2026-05-11T14:30:07,896 adding 'lm_eval/tasks/blimp/npi_present_2.yaml' 2026-05-11T14:30:07,897 adding 'lm_eval/tasks/blimp/only_npi_licensor_present.yaml' 2026-05-11T14:30:07,899 adding 'lm_eval/tasks/blimp/only_npi_scope.yaml' 2026-05-11T14:30:07,900 adding 'lm_eval/tasks/blimp/passive_1.yaml' 2026-05-11T14:30:07,901 adding 'lm_eval/tasks/blimp/passive_2.yaml' 2026-05-11T14:30:07,902 adding 'lm_eval/tasks/blimp/principle_A_c_command.yaml' 2026-05-11T14:30:07,903 adding 'lm_eval/tasks/blimp/principle_A_case_1.yaml' 2026-05-11T14:30:07,904 adding 'lm_eval/tasks/blimp/principle_A_case_2.yaml' 2026-05-11T14:30:07,905 adding 'lm_eval/tasks/blimp/principle_A_domain_1.yaml' 2026-05-11T14:30:07,907 adding 'lm_eval/tasks/blimp/principle_A_domain_2.yaml' 2026-05-11T14:30:07,908 adding 'lm_eval/tasks/blimp/principle_A_domain_3.yaml' 2026-05-11T14:30:07,909 adding 'lm_eval/tasks/blimp/principle_A_reconstruction.yaml' 2026-05-11T14:30:07,910 adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml' 2026-05-11T14:30:07,911 adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml' 2026-05-11T14:30:07,912 adding 'lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml' 2026-05-11T14:30:07,913 adding 'lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml' 2026-05-11T14:30:07,915 adding 'lm_eval/tasks/blimp/sentential_subject_island.yaml' 2026-05-11T14:30:07,916 adding 'lm_eval/tasks/blimp/superlative_quantifiers_1.yaml' 2026-05-11T14:30:07,917 adding 'lm_eval/tasks/blimp/superlative_quantifiers_2.yaml' 2026-05-11T14:30:07,918 adding 'lm_eval/tasks/blimp/tough_vs_raising_1.yaml' 2026-05-11T14:30:07,919 adding 'lm_eval/tasks/blimp/tough_vs_raising_2.yaml' 2026-05-11T14:30:07,920 adding 'lm_eval/tasks/blimp/transitive.yaml' 2026-05-11T14:30:07,921 adding 'lm_eval/tasks/blimp/wh_island.yaml' 2026-05-11T14:30:07,922 adding 'lm_eval/tasks/blimp/wh_questions_object_gap.yaml' 2026-05-11T14:30:07,923 adding 'lm_eval/tasks/blimp/wh_questions_subject_gap.yaml' 2026-05-11T14:30:07,924 adding 'lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml' 2026-05-11T14:30:07,925 adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml' 2026-05-11T14:30:07,926 adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml' 2026-05-11T14:30:07,928 adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml' 2026-05-11T14:30:07,929 adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml' 2026-05-11T14:30:07,932 adding 'lm_eval/tasks/blimp_nl/README.md' 2026-05-11T14:30:07,934 adding 'lm_eval/tasks/blimp_nl/_template_yaml' 2026-05-11T14:30:07,935 adding 'lm_eval/tasks/blimp_nl/adpositional_phrases__argument_r_extraction.yaml' 2026-05-11T14:30:07,936 adding 'lm_eval/tasks/blimp_nl/adpositional_phrases__argument_scrambling.yaml' 2026-05-11T14:30:07,937 adding 'lm_eval/tasks/blimp_nl/adverbial_modification__position_proform.yaml' 2026-05-11T14:30:07,938 adding 'lm_eval/tasks/blimp_nl/adverbial_modification__position_type.yaml' 2026-05-11T14:30:07,939 adding 'lm_eval/tasks/blimp_nl/anaphor_agreement__number.yaml' 2026-05-11T14:30:07,940 adding 'lm_eval/tasks/blimp_nl/anaphor_agreement__person.yaml' 2026-05-11T14:30:07,942 adding 'lm_eval/tasks/blimp_nl/argument_structure__argument_number_ditransitive.yaml' 2026-05-11T14:30:07,943 adding 'lm_eval/tasks/blimp_nl/argument_structure__argument_number_in_transitive.yaml' 2026-05-11T14:30:07,944 adding 'lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_1.yaml' 2026-05-11T14:30:07,945 adding 'lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_2.yaml' 2026-05-11T14:30:07,946 adding 'lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_3.yaml' 2026-05-11T14:30:07,947 adding 'lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_1.yaml' 2026-05-11T14:30:07,948 adding 'lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_2.yaml' 2026-05-11T14:30:07,950 adding 'lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_3.yaml' 2026-05-11T14:30:07,951 adding 'lm_eval/tasks/blimp_nl/auxiliaries__order_1.yaml' 2026-05-11T14:30:07,952 adding 'lm_eval/tasks/blimp_nl/auxiliaries__order_2.yaml' 2026-05-11T14:30:07,953 adding 'lm_eval/tasks/blimp_nl/auxiliaries__perfect.yaml' 2026-05-11T14:30:07,954 adding 'lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_1.yaml' 2026-05-11T14:30:07,955 adding 'lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_2.yaml' 2026-05-11T14:30:07,957 adding 'lm_eval/tasks/blimp_nl/binding_principle_a__c_command.yaml' 2026-05-11T14:30:07,958 adding 'lm_eval/tasks/blimp_nl/binding_principle_a__monomorphemic.yaml' 2026-05-11T14:30:07,959 adding 'lm_eval/tasks/blimp_nl/blimp_nl_group.yaml' 2026-05-11T14:30:07,960 adding 'lm_eval/tasks/blimp_nl/complementive__ditransitive.yaml' 2026-05-11T14:30:07,962 adding 'lm_eval/tasks/blimp_nl/complementive__intransitive.yaml' 2026-05-11T14:30:07,963 adding 'lm_eval/tasks/blimp_nl/complementive__position_adverb.yaml' 2026-05-11T14:30:07,964 adding 'lm_eval/tasks/blimp_nl/complementive__position_verb.yaml' 2026-05-11T14:30:07,965 adding 'lm_eval/tasks/blimp_nl/complementive__transitive.yaml' 2026-05-11T14:30:07,966 adding 'lm_eval/tasks/blimp_nl/crossing_dependencies__cross_dependency.yaml' 2026-05-11T14:30:07,967 adding 'lm_eval/tasks/blimp_nl/determiners__geen_expletive.yaml' 2026-05-11T14:30:07,968 adding 'lm_eval/tasks/blimp_nl/determiners__geen_scrambling_1.yaml' 2026-05-11T14:30:07,969 adding 'lm_eval/tasks/blimp_nl/determiners__geen_scrambling_2.yaml' 2026-05-11T14:30:07,970 adding 'lm_eval/tasks/blimp_nl/determiners__negative_polarity.yaml' 2026-05-11T14:30:07,971 adding 'lm_eval/tasks/blimp_nl/extraposition__adjectival_adverbial.yaml' 2026-05-11T14:30:07,972 adding 'lm_eval/tasks/blimp_nl/extraposition__adjectival_supplementive.yaml' 2026-05-11T14:30:07,973 adding 'lm_eval/tasks/blimp_nl/extraposition__argument_nominal.yaml' 2026-05-11T14:30:07,974 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__complementizer.yaml' 2026-05-11T14:30:07,975 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__perception_dat.yaml' 2026-05-11T14:30:07,977 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml' 2026-05-11T14:30:07,978 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml' 2026-05-11T14:30:07,979 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml' 2026-05-11T14:30:07,980 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml' 2026-05-11T14:30:07,981 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml' 2026-05-11T14:30:07,982 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml' 2026-05-11T14:30:07,983 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml' 2026-05-11T14:30:07,984 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml' 2026-05-11T14:30:07,986 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml' 2026-05-11T14:30:07,987 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml' 2026-05-11T14:30:07,989 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml' 2026-05-11T14:30:07,990 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml' 2026-05-11T14:30:07,991 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml' 2026-05-11T14:30:07,992 adding 'lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml' 2026-05-11T14:30:07,993 adding 'lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml' 2026-05-11T14:30:07,995 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml' 2026-05-11T14:30:07,996 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml' 2026-05-11T14:30:07,997 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml' 2026-05-11T14:30:07,998 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml' 2026-05-11T14:30:07,999 adding 'lm_eval/tasks/blimp_nl/passive__aci.yaml' 2026-05-11T14:30:08,000 adding 'lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml' 2026-05-11T14:30:08,001 adding 'lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml' 2026-05-11T14:30:08,003 adding 'lm_eval/tasks/blimp_nl/passive__impersonal.yaml' 2026-05-11T14:30:08,004 adding 'lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml' 2026-05-11T14:30:08,005 adding 'lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml' 2026-05-11T14:30:08,006 adding 'lm_eval/tasks/blimp_nl/r_words__adverbial.yaml' 2026-05-11T14:30:08,007 adding 'lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml' 2026-05-11T14:30:08,009 adding 'lm_eval/tasks/blimp_nl/relativization__island.yaml' 2026-05-11T14:30:08,010 adding 'lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml' 2026-05-11T14:30:08,011 adding 'lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml' 2026-05-11T14:30:08,012 adding 'lm_eval/tasks/blimp_nl/topicalization__island.yaml' 2026-05-11T14:30:08,013 adding 'lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml' 2026-05-11T14:30:08,014 adding 'lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml' 2026-05-11T14:30:08,015 adding 'lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml' 2026-05-11T14:30:08,016 adding 'lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml' 2026-05-11T14:30:08,018 adding 'lm_eval/tasks/blimp_nl/verb_second__order_main.yaml' 2026-05-11T14:30:08,019 adding 'lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml' 2026-05-11T14:30:08,020 adding 'lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml' 2026-05-11T14:30:08,021 adding 'lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml' 2026-05-11T14:30:08,022 adding 'lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml' 2026-05-11T14:30:08,023 adding 'lm_eval/tasks/blimp_nl/wh_movement__stranding_1.yaml' 2026-05-11T14:30:08,024 adding 'lm_eval/tasks/blimp_nl/wh_movement__stranding_2.yaml' 2026-05-11T14:30:08,025 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_1.yaml' 2026-05-11T14:30:08,026 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_2.yaml' 2026-05-11T14:30:08,027 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_1.yaml' 2026-05-11T14:30:08,028 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_2.yaml' 2026-05-11T14:30:08,029 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__resumptive_prolepsis.yaml' 2026-05-11T14:30:08,030 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__superiority.yaml' 2026-05-11T14:30:08,032 adding 'lm_eval/tasks/c4/README.md' 2026-05-11T14:30:08,033 adding 'lm_eval/tasks/c4/c4.yaml' 2026-05-11T14:30:08,034 adding 'lm_eval/tasks/c4/preprocess_c4.py' 2026-05-11T14:30:08,036 adding 'lm_eval/tasks/cabbq/README.md' 2026-05-11T14:30:08,037 adding 'lm_eval/tasks/cabbq/_cabbq_common_yaml' 2026-05-11T14:30:08,039 adding 'lm_eval/tasks/cabbq/cabbq.yaml' 2026-05-11T14:30:08,040 adding 'lm_eval/tasks/cabbq/cabbq_age.yaml' 2026-05-11T14:30:08,041 adding 'lm_eval/tasks/cabbq/cabbq_disability_status.yaml' 2026-05-11T14:30:08,042 adding 'lm_eval/tasks/cabbq/cabbq_gender.yaml' 2026-05-11T14:30:08,043 adding 'lm_eval/tasks/cabbq/cabbq_lgbtqia.yaml' 2026-05-11T14:30:08,044 adding 'lm_eval/tasks/cabbq/cabbq_nationality.yaml' 2026-05-11T14:30:08,045 adding 'lm_eval/tasks/cabbq/cabbq_physical_appearance.yaml' 2026-05-11T14:30:08,046 adding 'lm_eval/tasks/cabbq/cabbq_race_ethnicity.yaml' 2026-05-11T14:30:08,048 adding 'lm_eval/tasks/cabbq/cabbq_religion.yaml' 2026-05-11T14:30:08,049 adding 'lm_eval/tasks/cabbq/cabbq_ses.yaml' 2026-05-11T14:30:08,050 adding 'lm_eval/tasks/cabbq/cabbq_spanish_region.yaml' 2026-05-11T14:30:08,051 adding 'lm_eval/tasks/cabbq/utils.py' 2026-05-11T14:30:08,053 adding 'lm_eval/tasks/careqa/README.md' 2026-05-11T14:30:08,055 adding 'lm_eval/tasks/careqa/careqa_en.yaml' 2026-05-11T14:30:08,056 adding 'lm_eval/tasks/careqa/careqa_es.yaml' 2026-05-11T14:30:08,057 adding 'lm_eval/tasks/careqa/careqa_open.yaml' 2026-05-11T14:30:08,058 adding 'lm_eval/tasks/careqa/careqa_open_perplexity.yaml' 2026-05-11T14:30:08,059 adding 'lm_eval/tasks/careqa/utils.py' 2026-05-11T14:30:08,061 adding 'lm_eval/tasks/careqa/utils_open.py' 2026-05-11T14:30:08,062 adding 'lm_eval/tasks/careqa/utils_perplexity.py' 2026-05-11T14:30:08,065 adding 'lm_eval/tasks/catalan_bench/README.md' 2026-05-11T14:30:08,066 adding 'lm_eval/tasks/catalan_bench/_arc_ca_common_yaml' 2026-05-11T14:30:08,067 adding 'lm_eval/tasks/catalan_bench/_cabreu_common_yaml' 2026-05-11T14:30:08,068 adding 'lm_eval/tasks/catalan_bench/arc_ca_challenge.yaml' 2026-05-11T14:30:08,069 adding 'lm_eval/tasks/catalan_bench/arc_ca_easy.yaml' 2026-05-11T14:30:08,070 adding 'lm_eval/tasks/catalan_bench/cabreu_abstractive.yaml' 2026-05-11T14:30:08,071 adding 'lm_eval/tasks/catalan_bench/cabreu_extractive.yaml' 2026-05-11T14:30:08,073 adding 'lm_eval/tasks/catalan_bench/cabreu_extreme.yaml' 2026-05-11T14:30:08,074 adding 'lm_eval/tasks/catalan_bench/catalan_bench.yaml' 2026-05-11T14:30:08,075 adding 'lm_eval/tasks/catalan_bench/catalanqa.yaml' 2026-05-11T14:30:08,076 adding 'lm_eval/tasks/catalan_bench/catcola.yaml' 2026-05-11T14:30:08,077 adding 'lm_eval/tasks/catalan_bench/cocoteros_va.yaml' 2026-05-11T14:30:08,079 adding 'lm_eval/tasks/catalan_bench/copa_ca.yaml' 2026-05-11T14:30:08,080 adding 'lm_eval/tasks/catalan_bench/coqcat.yaml' 2026-05-11T14:30:08,081 adding 'lm_eval/tasks/catalan_bench/mgsm_direct_ca.yaml' 2026-05-11T14:30:08,082 adding 'lm_eval/tasks/catalan_bench/openbookqa_ca.yaml' 2026-05-11T14:30:08,083 adding 'lm_eval/tasks/catalan_bench/parafraseja.yaml' 2026-05-11T14:30:08,084 adding 'lm_eval/tasks/catalan_bench/paws_ca.yaml' 2026-05-11T14:30:08,085 adding 'lm_eval/tasks/catalan_bench/piqa_ca.yaml' 2026-05-11T14:30:08,086 adding 'lm_eval/tasks/catalan_bench/siqa_ca.yaml' 2026-05-11T14:30:08,087 adding 'lm_eval/tasks/catalan_bench/teca.yaml' 2026-05-11T14:30:08,089 adding 'lm_eval/tasks/catalan_bench/utils.py' 2026-05-11T14:30:08,090 adding 'lm_eval/tasks/catalan_bench/wnli_ca.yaml' 2026-05-11T14:30:08,091 adding 'lm_eval/tasks/catalan_bench/xnli_ca.yaml' 2026-05-11T14:30:08,092 adding 'lm_eval/tasks/catalan_bench/xnli_va.yaml' 2026-05-11T14:30:08,093 adding 'lm_eval/tasks/catalan_bench/xquad_ca.yaml' 2026-05-11T14:30:08,094 adding 'lm_eval/tasks/catalan_bench/xstorycloze_ca.yaml' 2026-05-11T14:30:08,096 adding 'lm_eval/tasks/catalan_bench/flores_ca/_flores_common_yaml' 2026-05-11T14:30:08,098 adding 'lm_eval/tasks/catalan_bench/flores_ca/create_yamls_flores_ca.py' 2026-05-11T14:30:08,099 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-de.yaml' 2026-05-11T14:30:08,100 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-en.yaml' 2026-05-11T14:30:08,101 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-es.yaml' 2026-05-11T14:30:08,102 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-eu.yaml' 2026-05-11T14:30:08,103 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-fr.yaml' 2026-05-11T14:30:08,104 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-gl.yaml' 2026-05-11T14:30:08,106 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-it.yaml' 2026-05-11T14:30:08,107 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-pt.yaml' 2026-05-11T14:30:08,108 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca.yaml' 2026-05-11T14:30:08,109 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_de-ca.yaml' 2026-05-11T14:30:08,110 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_en-ca.yaml' 2026-05-11T14:30:08,111 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_es-ca.yaml' 2026-05-11T14:30:08,113 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_eu-ca.yaml' 2026-05-11T14:30:08,114 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_fr-ca.yaml' 2026-05-11T14:30:08,115 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_gl-ca.yaml' 2026-05-11T14:30:08,116 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_it-ca.yaml' 2026-05-11T14:30:08,117 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_pt-ca.yaml' 2026-05-11T14:30:08,119 adding 'lm_eval/tasks/catalan_bench/phrases_va/_phrases_va_common' 2026-05-11T14:30:08,120 adding 'lm_eval/tasks/catalan_bench/phrases_va/phrases_ca-va.yaml' 2026-05-11T14:30:08,121 adding 'lm_eval/tasks/catalan_bench/phrases_va/phrases_va-ca.yaml' 2026-05-11T14:30:08,123 adding 'lm_eval/tasks/catalan_bench/truthfulqa_va/truthfulqa_va.yaml' 2026-05-11T14:30:08,125 adding 'lm_eval/tasks/catalan_bench/truthfulqa_va/utils.py' 2026-05-11T14:30:08,128 adding 'lm_eval/tasks/ceval/README.md' 2026-05-11T14:30:08,130 adding 'lm_eval/tasks/ceval/_ceval-valid.yaml' 2026-05-11T14:30:08,131 adding 'lm_eval/tasks/ceval/_default_ceval_yaml' 2026-05-11T14:30:08,132 adding 'lm_eval/tasks/ceval/_generate_configs.py' 2026-05-11T14:30:08,134 adding 'lm_eval/tasks/ceval/ceval-valid_accountant.yaml' 2026-05-11T14:30:08,135 adding 'lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml' 2026-05-11T14:30:08,136 adding 'lm_eval/tasks/ceval/ceval-valid_art_studies.yaml' 2026-05-11T14:30:08,137 adding 'lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml' 2026-05-11T14:30:08,138 adding 'lm_eval/tasks/ceval/ceval-valid_business_administration.yaml' 2026-05-11T14:30:08,139 adding 'lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml' 2026-05-11T14:30:08,140 adding 'lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml' 2026-05-11T14:30:08,142 adding 'lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml' 2026-05-11T14:30:08,143 adding 'lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml' 2026-05-11T14:30:08,144 adding 'lm_eval/tasks/ceval/ceval-valid_college_economics.yaml' 2026-05-11T14:30:08,145 adding 'lm_eval/tasks/ceval/ceval-valid_college_physics.yaml' 2026-05-11T14:30:08,146 adding 'lm_eval/tasks/ceval/ceval-valid_college_programming.yaml' 2026-05-11T14:30:08,147 adding 'lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml' 2026-05-11T14:30:08,148 adding 'lm_eval/tasks/ceval/ceval-valid_computer_network.yaml' 2026-05-11T14:30:08,149 adding 'lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml' 2026-05-11T14:30:08,150 adding 'lm_eval/tasks/ceval/ceval-valid_education_science.yaml' 2026-05-11T14:30:08,151 adding 'lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml' 2026-05-11T14:30:08,153 adding 'lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml' 2026-05-11T14:30:08,154 adding 'lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml' 2026-05-11T14:30:08,155 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml' 2026-05-11T14:30:08,156 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml' 2026-05-11T14:30:08,157 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml' 2026-05-11T14:30:08,158 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml' 2026-05-11T14:30:08,159 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml' 2026-05-11T14:30:08,160 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml' 2026-05-11T14:30:08,162 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml' 2026-05-11T14:30:08,163 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml' 2026-05-11T14:30:08,164 adding 'lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml' 2026-05-11T14:30:08,165 adding 'lm_eval/tasks/ceval/ceval-valid_law.yaml' 2026-05-11T14:30:08,166 adding 'lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml' 2026-05-11T14:30:08,168 adding 'lm_eval/tasks/ceval/ceval-valid_logic.yaml' 2026-05-11T14:30:08,169 adding 'lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml' 2026-05-11T14:30:08,170 adding 'lm_eval/tasks/ceval/ceval-valid_marxism.yaml' 2026-05-11T14:30:08,171 adding 'lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml' 2026-05-11T14:30:08,172 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml' 2026-05-11T14:30:08,174 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml' 2026-05-11T14:30:08,175 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml' 2026-05-11T14:30:08,176 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml' 2026-05-11T14:30:08,177 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml' 2026-05-11T14:30:08,178 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml' 2026-05-11T14:30:08,179 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml' 2026-05-11T14:30:08,181 adding 'lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml' 2026-05-11T14:30:08,182 adding 'lm_eval/tasks/ceval/ceval-valid_operating_system.yaml' 2026-05-11T14:30:08,185 adding 'lm_eval/tasks/ceval/ceval-valid_physician.yaml' 2026-05-11T14:30:08,186 adding 'lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml' 2026-05-11T14:30:08,187 adding 'lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml' 2026-05-11T14:30:08,188 adding 'lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml' 2026-05-11T14:30:08,189 adding 'lm_eval/tasks/ceval/ceval-valid_sports_science.yaml' 2026-05-11T14:30:08,191 adding 'lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml' 2026-05-11T14:30:08,192 adding 'lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml' 2026-05-11T14:30:08,193 adding 'lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml' 2026-05-11T14:30:08,194 adding 'lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml' 2026-05-11T14:30:08,196 adding 'lm_eval/tasks/chartqa/README.md' 2026-05-11T14:30:08,197 adding 'lm_eval/tasks/chartqa/chartqa.yaml' 2026-05-11T14:30:08,198 adding 'lm_eval/tasks/chartqa/chartqa_llama.yaml' 2026-05-11T14:30:08,199 adding 'lm_eval/tasks/chartqa/chartqa_llama_90.yaml' 2026-05-11T14:30:08,201 adding 'lm_eval/tasks/chartqa/utils.py' 2026-05-11T14:30:08,203 adding 'lm_eval/tasks/click/README.md' 2026-05-11T14:30:08,204 adding 'lm_eval/tasks/click/click.yaml' 2026-05-11T14:30:08,205 adding 'lm_eval/tasks/click/click_cul/_click_cul.yaml' 2026-05-11T14:30:08,206 adding 'lm_eval/tasks/click/click_cul/_default_click_cul_yaml' 2026-05-11T14:30:08,208 adding 'lm_eval/tasks/click/click_cul/click_cul_economy.yaml' 2026-05-11T14:30:08,209 adding 'lm_eval/tasks/click/click_cul/click_cul_geography.yaml' 2026-05-11T14:30:08,210 adding 'lm_eval/tasks/click/click_cul/click_cul_history.yaml' 2026-05-11T14:30:08,211 adding 'lm_eval/tasks/click/click_cul/click_cul_kpop.yaml' 2026-05-11T14:30:08,212 adding 'lm_eval/tasks/click/click_cul/click_cul_law.yaml' 2026-05-11T14:30:08,213 adding 'lm_eval/tasks/click/click_cul/click_cul_politics.yaml' 2026-05-11T14:30:08,214 adding 'lm_eval/tasks/click/click_cul/click_cul_society.yaml' 2026-05-11T14:30:08,215 adding 'lm_eval/tasks/click/click_cul/click_cul_tradition.yaml' 2026-05-11T14:30:08,217 adding 'lm_eval/tasks/click/click_cul/utils.py' 2026-05-11T14:30:08,218 adding 'lm_eval/tasks/click/click_lang/_click_lang.yaml' 2026-05-11T14:30:08,219 adding 'lm_eval/tasks/click/click_lang/_default_click_lang_yaml' 2026-05-11T14:30:08,221 adding 'lm_eval/tasks/click/click_lang/click_lang_function.yaml' 2026-05-11T14:30:08,222 adding 'lm_eval/tasks/click/click_lang/click_lang_grammar.yaml' 2026-05-11T14:30:08,223 adding 'lm_eval/tasks/click/click_lang/click_lang_text.yaml' 2026-05-11T14:30:08,224 adding 'lm_eval/tasks/click/click_lang/utils.py' 2026-05-11T14:30:08,227 adding 'lm_eval/tasks/cmmlu/README.md' 2026-05-11T14:30:08,228 adding 'lm_eval/tasks/cmmlu/_cmmlu.yaml' 2026-05-11T14:30:08,230 adding 'lm_eval/tasks/cmmlu/_default_template_yaml' 2026-05-11T14:30:08,231 adding 'lm_eval/tasks/cmmlu/_generate_configs.py' 2026-05-11T14:30:08,232 adding 'lm_eval/tasks/cmmlu/cmmlu_agronomy.yaml' 2026-05-11T14:30:08,234 adding 'lm_eval/tasks/cmmlu/cmmlu_anatomy.yaml' 2026-05-11T14:30:08,235 adding 'lm_eval/tasks/cmmlu/cmmlu_ancient_chinese.yaml' 2026-05-11T14:30:08,236 adding 'lm_eval/tasks/cmmlu/cmmlu_arts.yaml' 2026-05-11T14:30:08,237 adding 'lm_eval/tasks/cmmlu/cmmlu_astronomy.yaml' 2026-05-11T14:30:08,238 adding 'lm_eval/tasks/cmmlu/cmmlu_business_ethics.yaml' 2026-05-11T14:30:08,239 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_civil_service_exam.yaml' 2026-05-11T14:30:08,241 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_driving_rule.yaml' 2026-05-11T14:30:08,242 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_food_culture.yaml' 2026-05-11T14:30:08,243 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_foreign_policy.yaml' 2026-05-11T14:30:08,244 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_history.yaml' 2026-05-11T14:30:08,245 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_literature.yaml' 2026-05-11T14:30:08,247 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_teacher_qualification.yaml' 2026-05-11T14:30:08,248 adding 'lm_eval/tasks/cmmlu/cmmlu_clinical_knowledge.yaml' 2026-05-11T14:30:08,249 adding 'lm_eval/tasks/cmmlu/cmmlu_college_actuarial_science.yaml' 2026-05-11T14:30:08,250 adding 'lm_eval/tasks/cmmlu/cmmlu_college_education.yaml' 2026-05-11T14:30:08,251 adding 'lm_eval/tasks/cmmlu/cmmlu_college_engineering_hydrology.yaml' 2026-05-11T14:30:08,253 adding 'lm_eval/tasks/cmmlu/cmmlu_college_law.yaml' 2026-05-11T14:30:08,254 adding 'lm_eval/tasks/cmmlu/cmmlu_college_mathematics.yaml' 2026-05-11T14:30:08,255 adding 'lm_eval/tasks/cmmlu/cmmlu_college_medical_statistics.yaml' 2026-05-11T14:30:08,256 adding 'lm_eval/tasks/cmmlu/cmmlu_college_medicine.yaml' 2026-05-11T14:30:08,258 adding 'lm_eval/tasks/cmmlu/cmmlu_computer_science.yaml' 2026-05-11T14:30:08,259 adding 'lm_eval/tasks/cmmlu/cmmlu_computer_security.yaml' 2026-05-11T14:30:08,260 adding 'lm_eval/tasks/cmmlu/cmmlu_conceptual_physics.yaml' 2026-05-11T14:30:08,261 adding 'lm_eval/tasks/cmmlu/cmmlu_construction_project_management.yaml' 2026-05-11T14:30:08,262 adding 'lm_eval/tasks/cmmlu/cmmlu_economics.yaml' 2026-05-11T14:30:08,263 adding 'lm_eval/tasks/cmmlu/cmmlu_education.yaml' 2026-05-11T14:30:08,264 adding 'lm_eval/tasks/cmmlu/cmmlu_electrical_engineering.yaml' 2026-05-11T14:30:08,265 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_chinese.yaml' 2026-05-11T14:30:08,266 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_commonsense.yaml' 2026-05-11T14:30:08,267 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_information_and_technology.yaml' 2026-05-11T14:30:08,269 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_mathematics.yaml' 2026-05-11T14:30:08,270 adding 'lm_eval/tasks/cmmlu/cmmlu_ethnology.yaml' 2026-05-11T14:30:08,271 adding 'lm_eval/tasks/cmmlu/cmmlu_food_science.yaml' 2026-05-11T14:30:08,273 adding 'lm_eval/tasks/cmmlu/cmmlu_genetics.yaml' 2026-05-11T14:30:08,274 adding 'lm_eval/tasks/cmmlu/cmmlu_global_facts.yaml' 2026-05-11T14:30:08,275 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_biology.yaml' 2026-05-11T14:30:08,276 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_chemistry.yaml' 2026-05-11T14:30:08,277 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_geography.yaml' 2026-05-11T14:30:08,279 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_mathematics.yaml' 2026-05-11T14:30:08,280 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_physics.yaml' 2026-05-11T14:30:08,281 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_politics.yaml' 2026-05-11T14:30:08,282 adding 'lm_eval/tasks/cmmlu/cmmlu_human_sexuality.yaml' 2026-05-11T14:30:08,284 adding 'lm_eval/tasks/cmmlu/cmmlu_international_law.yaml' 2026-05-11T14:30:08,285 adding 'lm_eval/tasks/cmmlu/cmmlu_journalism.yaml' 2026-05-11T14:30:08,286 adding 'lm_eval/tasks/cmmlu/cmmlu_jurisprudence.yaml' 2026-05-11T14:30:08,287 adding 'lm_eval/tasks/cmmlu/cmmlu_legal_and_moral_basis.yaml' 2026-05-11T14:30:08,289 adding 'lm_eval/tasks/cmmlu/cmmlu_logical.yaml' 2026-05-11T14:30:08,290 adding 'lm_eval/tasks/cmmlu/cmmlu_machine_learning.yaml' 2026-05-11T14:30:08,291 adding 'lm_eval/tasks/cmmlu/cmmlu_management.yaml' 2026-05-11T14:30:08,292 adding 'lm_eval/tasks/cmmlu/cmmlu_marketing.yaml' 2026-05-11T14:30:08,294 adding 'lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml' 2026-05-11T14:30:08,295 adding 'lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml' 2026-05-11T14:30:08,296 adding 'lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml' 2026-05-11T14:30:08,297 adding 'lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml' 2026-05-11T14:30:08,298 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml' 2026-05-11T14:30:08,300 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml' 2026-05-11T14:30:08,301 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml' 2026-05-11T14:30:08,302 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml' 2026-05-11T14:30:08,303 adding 'lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml' 2026-05-11T14:30:08,305 adding 'lm_eval/tasks/cmmlu/cmmlu_security_study.yaml' 2026-05-11T14:30:08,306 adding 'lm_eval/tasks/cmmlu/cmmlu_sociology.yaml' 2026-05-11T14:30:08,307 adding 'lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml' 2026-05-11T14:30:08,308 adding 'lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml' 2026-05-11T14:30:08,309 adding 'lm_eval/tasks/cmmlu/cmmlu_virology.yaml' 2026-05-11T14:30:08,311 adding 'lm_eval/tasks/cmmlu/cmmlu_world_history.yaml' 2026-05-11T14:30:08,312 adding 'lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml' 2026-05-11T14:30:08,314 adding 'lm_eval/tasks/cnn_dailymail/README.md' 2026-05-11T14:30:08,315 adding 'lm_eval/tasks/cnn_dailymail/cnn_dailymail.yaml' 2026-05-11T14:30:08,317 adding 'lm_eval/tasks/cnn_dailymail/utils.py' 2026-05-11T14:30:08,319 adding 'lm_eval/tasks/code_x_glue/code-text/README.md' 2026-05-11T14:30:08,320 adding 'lm_eval/tasks/code_x_glue/code-text/_codexglue.yaml' 2026-05-11T14:30:08,321 adding 'lm_eval/tasks/code_x_glue/code-text/_default_template_yaml' 2026-05-11T14:30:08,323 adding 'lm_eval/tasks/code_x_glue/code-text/bleu.py' 2026-05-11T14:30:08,324 adding 'lm_eval/tasks/code_x_glue/code-text/go.yaml' 2026-05-11T14:30:08,325 adding 'lm_eval/tasks/code_x_glue/code-text/java.yaml' 2026-05-11T14:30:08,326 adding 'lm_eval/tasks/code_x_glue/code-text/javascript.yaml' 2026-05-11T14:30:08,327 adding 'lm_eval/tasks/code_x_glue/code-text/php.yaml' 2026-05-11T14:30:08,328 adding 'lm_eval/tasks/code_x_glue/code-text/python.yaml' 2026-05-11T14:30:08,329 adding 'lm_eval/tasks/code_x_glue/code-text/ruby.yaml' 2026-05-11T14:30:08,331 adding 'lm_eval/tasks/code_x_glue/code-text/utils.py' 2026-05-11T14:30:08,332 adding 'lm_eval/tasks/common_voice/common_voice_en.yaml' 2026-05-11T14:30:08,334 adding 'lm_eval/tasks/common_voice/utils.py' 2026-05-11T14:30:08,335 adding 'lm_eval/tasks/commonsense_qa/README.md' 2026-05-11T14:30:08,336 adding 'lm_eval/tasks/commonsense_qa/default.yaml' 2026-05-11T14:30:08,338 adding 'lm_eval/tasks/copal_id/README.md' 2026-05-11T14:30:08,339 adding 'lm_eval/tasks/copal_id/colloquial.yaml' 2026-05-11T14:30:08,341 adding 'lm_eval/tasks/copal_id/standard.yaml' 2026-05-11T14:30:08,342 adding 'lm_eval/tasks/copal_id/utils.py' 2026-05-11T14:30:08,343 adding 'lm_eval/tasks/coqa/README.md' 2026-05-11T14:30:08,345 adding 'lm_eval/tasks/coqa/default.yaml' 2026-05-11T14:30:08,346 adding 'lm_eval/tasks/coqa/utils.py' 2026-05-11T14:30:08,349 adding 'lm_eval/tasks/crows_pairs/README.md' 2026-05-11T14:30:08,350 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english.yaml' 2026-05-11T14:30:08,351 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml' 2026-05-11T14:30:08,352 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml' 2026-05-11T14:30:08,353 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml' 2026-05-11T14:30:08,354 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml' 2026-05-11T14:30:08,355 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml' 2026-05-11T14:30:08,356 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml' 2026-05-11T14:30:08,358 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml' 2026-05-11T14:30:08,359 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml' 2026-05-11T14:30:08,360 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml' 2026-05-11T14:30:08,361 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml' 2026-05-11T14:30:08,362 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french.yaml' 2026-05-11T14:30:08,363 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml' 2026-05-11T14:30:08,364 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml' 2026-05-11T14:30:08,365 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml' 2026-05-11T14:30:08,366 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml' 2026-05-11T14:30:08,368 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml' 2026-05-11T14:30:08,369 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml' 2026-05-11T14:30:08,370 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml' 2026-05-11T14:30:08,371 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml' 2026-05-11T14:30:08,372 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml' 2026-05-11T14:30:08,373 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml' 2026-05-11T14:30:08,375 adding 'lm_eval/tasks/crows_pairs/utils.py' 2026-05-11T14:30:08,377 adding 'lm_eval/tasks/cruxeval/README.md' 2026-05-11T14:30:08,378 adding 'lm_eval/tasks/cruxeval/cruxeval_common_yaml' 2026-05-11T14:30:08,379 adding 'lm_eval/tasks/cruxeval/cruxeval_input.yaml' 2026-05-11T14:30:08,380 adding 'lm_eval/tasks/cruxeval/cruxeval_input_08.yaml' 2026-05-11T14:30:08,381 adding 'lm_eval/tasks/cruxeval/cruxeval_input_cot.yaml' 2026-05-11T14:30:08,382 adding 'lm_eval/tasks/cruxeval/cruxeval_output.yaml' 2026-05-11T14:30:08,383 adding 'lm_eval/tasks/cruxeval/cruxeval_output_08.yaml' 2026-05-11T14:30:08,384 adding 'lm_eval/tasks/cruxeval/cruxeval_output_cot.yaml' 2026-05-11T14:30:08,386 adding 'lm_eval/tasks/cruxeval/utils.py' 2026-05-11T14:30:08,388 adding 'lm_eval/tasks/csatqa/_csatqa.yaml' 2026-05-11T14:30:08,389 adding 'lm_eval/tasks/csatqa/_default_csatqa_yaml' 2026-05-11T14:30:08,390 adding 'lm_eval/tasks/csatqa/_generate_configs.py' 2026-05-11T14:30:08,391 adding 'lm_eval/tasks/csatqa/csatqa_gr.yaml' 2026-05-11T14:30:08,393 adding 'lm_eval/tasks/csatqa/csatqa_li.yaml' 2026-05-11T14:30:08,394 adding 'lm_eval/tasks/csatqa/csatqa_rch.yaml' 2026-05-11T14:30:08,395 adding 'lm_eval/tasks/csatqa/csatqa_rcs.yaml' 2026-05-11T14:30:08,396 adding 'lm_eval/tasks/csatqa/csatqa_rcss.yaml' 2026-05-11T14:30:08,397 adding 'lm_eval/tasks/csatqa/csatqa_wr.yaml' 2026-05-11T14:30:08,398 adding 'lm_eval/tasks/csatqa/utils.py' 2026-05-11T14:30:08,400 adding 'lm_eval/tasks/darija_bench/README.md' 2026-05-11T14:30:08,402 adding 'lm_eval/tasks/darija_bench/darija_sentiment/README.md' 2026-05-11T14:30:08,403 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment.yaml' 2026-05-11T14:30:08,404 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_electrom.yaml' 2026-05-11T14:30:08,405 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_mac.yaml' 2026-05-11T14:30:08,407 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msac.yaml' 2026-05-11T14:30:08,408 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msda.yaml' 2026-05-11T14:30:08,409 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_myc.yaml' 2026-05-11T14:30:08,410 adding 'lm_eval/tasks/darija_bench/darija_sentiment/default_darija_sentiment_template_yaml' 2026-05-11T14:30:08,412 adding 'lm_eval/tasks/darija_bench/darija_sentiment/utils.py' 2026-05-11T14:30:08,414 adding 'lm_eval/tasks/darija_bench/darija_summarization/README.md' 2026-05-11T14:30:08,415 adding 'lm_eval/tasks/darija_bench/darija_summarization/summarization.yaml' 2026-05-11T14:30:08,416 adding 'lm_eval/tasks/darija_bench/darija_summarization/summarization_common_yaml' 2026-05-11T14:30:08,417 adding 'lm_eval/tasks/darija_bench/darija_summarization/summarization_darija.yaml' 2026-05-11T14:30:08,418 adding 'lm_eval/tasks/darija_bench/darija_summarization/utils.py' 2026-05-11T14:30:08,421 adding 'lm_eval/tasks/darija_bench/darija_translation/README.md' 2026-05-11T14:30:08,422 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_common_yaml' 2026-05-11T14:30:08,423 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_all.yaml' 2026-05-11T14:30:08,424 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_darija.yaml' 2026-05-11T14:30:08,426 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_en.yaml' 2026-05-11T14:30:08,427 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_fr.yaml' 2026-05-11T14:30:08,428 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_msa.yaml' 2026-05-11T14:30:08,429 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_en_dr.yaml' 2026-05-11T14:30:08,430 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_fr_dr.yaml' 2026-05-11T14:30:08,432 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_msa_dr.yaml' 2026-05-11T14:30:08,433 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_common_yaml' 2026-05-11T14:30:08,434 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_all.yaml' 2026-05-11T14:30:08,435 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_darija.yaml' 2026-05-11T14:30:08,436 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_en.yaml' 2026-05-11T14:30:08,437 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_fr.yaml' 2026-05-11T14:30:08,438 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_msa.yaml' 2026-05-11T14:30:08,439 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_en_dr.yaml' 2026-05-11T14:30:08,440 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_fr_dr.yaml' 2026-05-11T14:30:08,441 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_msa_dr.yaml' 2026-05-11T14:30:08,442 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_common_yaml' 2026-05-11T14:30:08,444 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_all.yaml' 2026-05-11T14:30:08,445 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_darija.yaml' 2026-05-11T14:30:08,446 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_dr_msa.yaml' 2026-05-11T14:30:08,447 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_msa_dr.yaml' 2026-05-11T14:30:08,448 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_common_yaml' 2026-05-11T14:30:08,449 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_all.yaml' 2026-05-11T14:30:08,450 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_darija.yaml' 2026-05-11T14:30:08,451 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_dr_en.yaml' 2026-05-11T14:30:08,453 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_en_dr.yaml' 2026-05-11T14:30:08,454 adding 'lm_eval/tasks/darija_bench/darija_translation/translation_common_yaml' 2026-05-11T14:30:08,455 adding 'lm_eval/tasks/darija_bench/darija_translation/translation_darija.yaml' 2026-05-11T14:30:08,456 adding 'lm_eval/tasks/darija_bench/darija_translation/utils.py' 2026-05-11T14:30:08,458 adding 'lm_eval/tasks/darija_bench/darija_transliteration/README.md' 2026-05-11T14:30:08,460 adding 'lm_eval/tasks/darija_bench/darija_transliteration/translation_ar_dr.yaml' 2026-05-11T14:30:08,461 adding 'lm_eval/tasks/darija_bench/darija_transliteration/translation_dr_ar.yaml' 2026-05-11T14:30:08,462 adding 'lm_eval/tasks/darija_bench/darija_transliteration/transliteration_all.yaml' 2026-05-11T14:30:08,463 adding 'lm_eval/tasks/darija_bench/darija_transliteration/transliteration_common_yaml' 2026-05-11T14:30:08,464 adding 'lm_eval/tasks/darija_bench/darija_transliteration/transliteration_darija.yaml' 2026-05-11T14:30:08,466 adding 'lm_eval/tasks/darija_bench/darija_transliteration/utils.py' 2026-05-11T14:30:08,467 adding 'lm_eval/tasks/darijahellaswag/README.md' 2026-05-11T14:30:08,469 adding 'lm_eval/tasks/darijahellaswag/darijahellaswag.yaml' 2026-05-11T14:30:08,470 adding 'lm_eval/tasks/darijahellaswag/utils.py' 2026-05-11T14:30:08,473 adding 'lm_eval/tasks/darijammlu/README.md' 2026-05-11T14:30:08,474 adding 'lm_eval/tasks/darijammlu/_darijammlu.yaml' 2026-05-11T14:30:08,475 adding 'lm_eval/tasks/darijammlu/_darijammlu_ar_mmlu.yaml' 2026-05-11T14:30:08,476 adding 'lm_eval/tasks/darijammlu/_darijammlu_mmlu.yaml' 2026-05-11T14:30:08,477 adding 'lm_eval/tasks/darijammlu/_default_darijammlu_template_yaml' 2026-05-11T14:30:08,479 adding 'lm_eval/tasks/darijammlu/_generate_configs.py' 2026-05-11T14:30:08,480 adding 'lm_eval/tasks/darijammlu/darijammlu_accounting.yaml' 2026-05-11T14:30:08,481 adding 'lm_eval/tasks/darijammlu/darijammlu_arabic_language.yaml' 2026-05-11T14:30:08,482 adding 'lm_eval/tasks/darijammlu/darijammlu_arabic_language_(general).yaml' 2026-05-11T14:30:08,484 adding 'lm_eval/tasks/darijammlu/darijammlu_arabic_language_(grammar).yaml' 2026-05-11T14:30:08,485 adding 'lm_eval/tasks/darijammlu/darijammlu_biology.yaml' 2026-05-11T14:30:08,486 adding 'lm_eval/tasks/darijammlu/darijammlu_civics.yaml' 2026-05-11T14:30:08,487 adding 'lm_eval/tasks/darijammlu/darijammlu_computer_science.yaml' 2026-05-11T14:30:08,488 adding 'lm_eval/tasks/darijammlu/darijammlu_driving_test.yaml' 2026-05-11T14:30:08,489 adding 'lm_eval/tasks/darijammlu/darijammlu_economics.yaml' 2026-05-11T14:30:08,490 adding 'lm_eval/tasks/darijammlu/darijammlu_general_knowledge.yaml' 2026-05-11T14:30:08,491 adding 'lm_eval/tasks/darijammlu/darijammlu_geography.yaml' 2026-05-11T14:30:08,492 adding 'lm_eval/tasks/darijammlu/darijammlu_global_facts.yaml' 2026-05-11T14:30:08,493 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_european_history.yaml' 2026-05-11T14:30:08,494 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_geography.yaml' 2026-05-11T14:30:08,496 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:08,497 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_psychology.yaml' 2026-05-11T14:30:08,498 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_statistics.yaml' 2026-05-11T14:30:08,499 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_world_history.yaml' 2026-05-11T14:30:08,501 adding 'lm_eval/tasks/darijammlu/darijammlu_history.yaml' 2026-05-11T14:30:08,502 adding 'lm_eval/tasks/darijammlu/darijammlu_human_aging.yaml' 2026-05-11T14:30:08,503 adding 'lm_eval/tasks/darijammlu/darijammlu_international_law.yaml' 2026-05-11T14:30:08,504 adding 'lm_eval/tasks/darijammlu/darijammlu_islamic_studies.yaml' 2026-05-11T14:30:08,505 adding 'lm_eval/tasks/darijammlu/darijammlu_jurisprudence.yaml' 2026-05-11T14:30:08,506 adding 'lm_eval/tasks/darijammlu/darijammlu_law.yaml' 2026-05-11T14:30:08,508 adding 'lm_eval/tasks/darijammlu/darijammlu_logical_fallacies.yaml' 2026-05-11T14:30:08,509 adding 'lm_eval/tasks/darijammlu/darijammlu_management.yaml' 2026-05-11T14:30:08,510 adding 'lm_eval/tasks/darijammlu/darijammlu_management_ar.yaml' 2026-05-11T14:30:08,511 adding 'lm_eval/tasks/darijammlu/darijammlu_marketing.yaml' 2026-05-11T14:30:08,512 adding 'lm_eval/tasks/darijammlu/darijammlu_math.yaml' 2026-05-11T14:30:08,513 adding 'lm_eval/tasks/darijammlu/darijammlu_moral_disputes.yaml' 2026-05-11T14:30:08,514 adding 'lm_eval/tasks/darijammlu/darijammlu_moral_scenarios.yaml' 2026-05-11T14:30:08,516 adding 'lm_eval/tasks/darijammlu/darijammlu_natural_science.yaml' 2026-05-11T14:30:08,517 adding 'lm_eval/tasks/darijammlu/darijammlu_nutrition.yaml' 2026-05-11T14:30:08,518 adding 'lm_eval/tasks/darijammlu/darijammlu_philosophy.yaml' 2026-05-11T14:30:08,519 adding 'lm_eval/tasks/darijammlu/darijammlu_philosophy_ar.yaml' 2026-05-11T14:30:08,520 adding 'lm_eval/tasks/darijammlu/darijammlu_physics.yaml' 2026-05-11T14:30:08,522 adding 'lm_eval/tasks/darijammlu/darijammlu_political_science.yaml' 2026-05-11T14:30:08,523 adding 'lm_eval/tasks/darijammlu/darijammlu_professional_law.yaml' 2026-05-11T14:30:08,524 adding 'lm_eval/tasks/darijammlu/darijammlu_professional_psychology.yaml' 2026-05-11T14:30:08,525 adding 'lm_eval/tasks/darijammlu/darijammlu_public_relations.yaml' 2026-05-11T14:30:08,527 adding 'lm_eval/tasks/darijammlu/darijammlu_security_studies.yaml' 2026-05-11T14:30:08,528 adding 'lm_eval/tasks/darijammlu/darijammlu_social_science.yaml' 2026-05-11T14:30:08,529 adding 'lm_eval/tasks/darijammlu/darijammlu_sociology.yaml' 2026-05-11T14:30:08,530 adding 'lm_eval/tasks/darijammlu/darijammlu_world_religions.yaml' 2026-05-11T14:30:08,531 adding 'lm_eval/tasks/darijammlu/utils.py' 2026-05-11T14:30:08,533 adding 'lm_eval/tasks/discrim_eval/README.md' 2026-05-11T14:30:08,535 adding 'lm_eval/tasks/discrim_eval/discrim_eval_explicit.yaml' 2026-05-11T14:30:08,536 adding 'lm_eval/tasks/discrim_eval/discrim_eval_implicit.yaml' 2026-05-11T14:30:08,538 adding 'lm_eval/tasks/discrim_eval/utils.py' 2026-05-11T14:30:08,540 adding 'lm_eval/tasks/drop/README.md' 2026-05-11T14:30:08,541 adding 'lm_eval/tasks/drop/default.yaml' 2026-05-11T14:30:08,543 adding 'lm_eval/tasks/drop/utils.py' 2026-05-11T14:30:08,545 adding 'lm_eval/tasks/e2lmc/mmlu_early_training/README.md' 2026-05-11T14:30:08,547 adding 'lm_eval/tasks/e2lmc/mmlu_early_training/custom_metrics.py' 2026-05-11T14:30:08,548 adding 'lm_eval/tasks/e2lmc/mmlu_early_training/mmlu_early_training.yaml' 2026-05-11T14:30:08,550 adding 'lm_eval/tasks/e2lmc/noor/README.md' 2026-05-11T14:30:08,552 adding 'lm_eval/tasks/e2lmc/noor/_generate_configs.py' 2026-05-11T14:30:08,553 adding 'lm_eval/tasks/e2lmc/noor/_noor.yaml' 2026-05-11T14:30:08,554 adding 'lm_eval/tasks/e2lmc/noor/_noor_stem.yaml' 2026-05-11T14:30:08,555 adding 'lm_eval/tasks/e2lmc/noor/_noor_template' 2026-05-11T14:30:08,556 adding 'lm_eval/tasks/e2lmc/noor/noor_abstract_algebra.yaml' 2026-05-11T14:30:08,558 adding 'lm_eval/tasks/e2lmc/noor/noor_college_computer_science.yaml' 2026-05-11T14:30:08,559 adding 'lm_eval/tasks/e2lmc/noor/noor_college_mathematics.yaml' 2026-05-11T14:30:08,560 adding 'lm_eval/tasks/e2lmc/noor/noor_college_physics.yaml' 2026-05-11T14:30:08,561 adding 'lm_eval/tasks/e2lmc/noor/noor_conceptual_physics.yaml' 2026-05-11T14:30:08,562 adding 'lm_eval/tasks/e2lmc/noor/noor_electrical_engineering.yaml' 2026-05-11T14:30:08,563 adding 'lm_eval/tasks/e2lmc/noor/noor_elementary_mathematics.yaml' 2026-05-11T14:30:08,564 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_computer_science.yaml' 2026-05-11T14:30:08,566 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_mathematics.yaml' 2026-05-11T14:30:08,567 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_physics.yaml' 2026-05-11T14:30:08,568 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_statistics.yaml' 2026-05-11T14:30:08,569 adding 'lm_eval/tasks/e2lmc/noor/noor_machine_learning.yaml' 2026-05-11T14:30:08,571 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/README.md' 2026-05-11T14:30:08,572 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/_sciknoweval_mcqa.yaml' 2026-05-11T14:30:08,573 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/_var5shots_template_yaml' 2026-05-11T14:30:08,574 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Biology.yaml' 2026-05-11T14:30:08,576 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Chemistry.yaml' 2026-05-11T14:30:08,577 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Material.yaml' 2026-05-11T14:30:08,578 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Physics.yaml' 2026-05-11T14:30:08,580 adding 'lm_eval/tasks/egyhellaswag/README.md' 2026-05-11T14:30:08,581 adding 'lm_eval/tasks/egyhellaswag/egyhellaswag.yaml' 2026-05-11T14:30:08,582 adding 'lm_eval/tasks/egyhellaswag/utils.py' 2026-05-11T14:30:08,585 adding 'lm_eval/tasks/egymmlu/README.md' 2026-05-11T14:30:08,586 adding 'lm_eval/tasks/egymmlu/_default_egymmlu_template_yaml' 2026-05-11T14:30:08,587 adding 'lm_eval/tasks/egymmlu/_egymmlu.yaml' 2026-05-11T14:30:08,588 adding 'lm_eval/tasks/egymmlu/_egymmlu_ar_mmlu.yaml' 2026-05-11T14:30:08,589 adding 'lm_eval/tasks/egymmlu/_egymmlu_mmlu.yaml' 2026-05-11T14:30:08,591 adding 'lm_eval/tasks/egymmlu/_generate_configs.py' 2026-05-11T14:30:08,592 adding 'lm_eval/tasks/egymmlu/egymmlu_accounting.yaml' 2026-05-11T14:30:08,593 adding 'lm_eval/tasks/egymmlu/egymmlu_arabic_language.yaml' 2026-05-11T14:30:08,595 adding 'lm_eval/tasks/egymmlu/egymmlu_arabic_language_(general).yaml' 2026-05-11T14:30:08,596 adding 'lm_eval/tasks/egymmlu/egymmlu_arabic_language_(grammar).yaml' 2026-05-11T14:30:08,597 adding 'lm_eval/tasks/egymmlu/egymmlu_biology.yaml' 2026-05-11T14:30:08,598 adding 'lm_eval/tasks/egymmlu/egymmlu_civics.yaml' 2026-05-11T14:30:08,599 adding 'lm_eval/tasks/egymmlu/egymmlu_computer_science.yaml' 2026-05-11T14:30:08,600 adding 'lm_eval/tasks/egymmlu/egymmlu_driving_test.yaml' 2026-05-11T14:30:08,601 adding 'lm_eval/tasks/egymmlu/egymmlu_economics.yaml' 2026-05-11T14:30:08,603 adding 'lm_eval/tasks/egymmlu/egymmlu_general_knowledge.yaml' 2026-05-11T14:30:08,604 adding 'lm_eval/tasks/egymmlu/egymmlu_geography.yaml' 2026-05-11T14:30:08,605 adding 'lm_eval/tasks/egymmlu/egymmlu_global_facts.yaml' 2026-05-11T14:30:08,606 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_european_history.yaml' 2026-05-11T14:30:08,607 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_geography.yaml' 2026-05-11T14:30:08,608 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:08,609 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_psychology.yaml' 2026-05-11T14:30:08,610 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_statistics.yaml' 2026-05-11T14:30:08,611 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_world_history.yaml' 2026-05-11T14:30:08,612 adding 'lm_eval/tasks/egymmlu/egymmlu_history.yaml' 2026-05-11T14:30:08,613 adding 'lm_eval/tasks/egymmlu/egymmlu_human_aging.yaml' 2026-05-11T14:30:08,614 adding 'lm_eval/tasks/egymmlu/egymmlu_international_law.yaml' 2026-05-11T14:30:08,615 adding 'lm_eval/tasks/egymmlu/egymmlu_islamic_studies.yaml' 2026-05-11T14:30:08,617 adding 'lm_eval/tasks/egymmlu/egymmlu_jurisprudence.yaml' 2026-05-11T14:30:08,618 adding 'lm_eval/tasks/egymmlu/egymmlu_law.yaml' 2026-05-11T14:30:08,619 adding 'lm_eval/tasks/egymmlu/egymmlu_logical_fallacies.yaml' 2026-05-11T14:30:08,620 adding 'lm_eval/tasks/egymmlu/egymmlu_management.yaml' 2026-05-11T14:30:08,621 adding 'lm_eval/tasks/egymmlu/egymmlu_management_ar.yaml' 2026-05-11T14:30:08,622 adding 'lm_eval/tasks/egymmlu/egymmlu_marketing.yaml' 2026-05-11T14:30:08,623 adding 'lm_eval/tasks/egymmlu/egymmlu_math.yaml' 2026-05-11T14:30:08,625 adding 'lm_eval/tasks/egymmlu/egymmlu_moral_disputes.yaml' 2026-05-11T14:30:08,626 adding 'lm_eval/tasks/egymmlu/egymmlu_moral_scenarios.yaml' 2026-05-11T14:30:08,627 adding 'lm_eval/tasks/egymmlu/egymmlu_natural_science.yaml' 2026-05-11T14:30:08,628 adding 'lm_eval/tasks/egymmlu/egymmlu_nutrition.yaml' 2026-05-11T14:30:08,629 adding 'lm_eval/tasks/egymmlu/egymmlu_philosophy.yaml' 2026-05-11T14:30:08,630 adding 'lm_eval/tasks/egymmlu/egymmlu_philosophy_ar.yaml' 2026-05-11T14:30:08,631 adding 'lm_eval/tasks/egymmlu/egymmlu_physics.yaml' 2026-05-11T14:30:08,632 adding 'lm_eval/tasks/egymmlu/egymmlu_political_science.yaml' 2026-05-11T14:30:08,634 adding 'lm_eval/tasks/egymmlu/egymmlu_professional_law.yaml' 2026-05-11T14:30:08,635 adding 'lm_eval/tasks/egymmlu/egymmlu_professional_psychology.yaml' 2026-05-11T14:30:08,636 adding 'lm_eval/tasks/egymmlu/egymmlu_public_relations.yaml' 2026-05-11T14:30:08,637 adding 'lm_eval/tasks/egymmlu/egymmlu_security_studies.yaml' 2026-05-11T14:30:08,638 adding 'lm_eval/tasks/egymmlu/egymmlu_social_science.yaml' 2026-05-11T14:30:08,639 adding 'lm_eval/tasks/egymmlu/egymmlu_sociology.yaml' 2026-05-11T14:30:08,640 adding 'lm_eval/tasks/egymmlu/egymmlu_world_religions.yaml' 2026-05-11T14:30:08,642 adding 'lm_eval/tasks/egymmlu/utils.py' 2026-05-11T14:30:08,644 adding 'lm_eval/tasks/eq_bench/README.md' 2026-05-11T14:30:08,645 adding 'lm_eval/tasks/eq_bench/default.yaml' 2026-05-11T14:30:08,646 adding 'lm_eval/tasks/eq_bench/utils.py' 2026-05-11T14:30:08,648 adding 'lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml' 2026-05-11T14:30:08,649 adding 'lm_eval/tasks/eq_bench/multilingual/eqbench_es.yaml' 2026-05-11T14:30:08,650 adding 'lm_eval/tasks/eq_bench/multilingual/utils.py' 2026-05-11T14:30:08,652 adding 'lm_eval/tasks/esbbq/README.md' 2026-05-11T14:30:08,654 adding 'lm_eval/tasks/esbbq/_esbbq_common_yaml' 2026-05-11T14:30:08,655 adding 'lm_eval/tasks/esbbq/esbbq.yaml' 2026-05-11T14:30:08,656 adding 'lm_eval/tasks/esbbq/esbbq_age.yaml' 2026-05-11T14:30:08,657 adding 'lm_eval/tasks/esbbq/esbbq_disability_status.yaml' 2026-05-11T14:30:08,658 adding 'lm_eval/tasks/esbbq/esbbq_gender.yaml' 2026-05-11T14:30:08,659 adding 'lm_eval/tasks/esbbq/esbbq_lgbtqia.yaml' 2026-05-11T14:30:08,660 adding 'lm_eval/tasks/esbbq/esbbq_nationality.yaml' 2026-05-11T14:30:08,662 adding 'lm_eval/tasks/esbbq/esbbq_physical_appearance.yaml' 2026-05-11T14:30:08,663 adding 'lm_eval/tasks/esbbq/esbbq_race_ethnicity.yaml' 2026-05-11T14:30:08,664 adding 'lm_eval/tasks/esbbq/esbbq_religion.yaml' 2026-05-11T14:30:08,665 adding 'lm_eval/tasks/esbbq/esbbq_ses.yaml' 2026-05-11T14:30:08,666 adding 'lm_eval/tasks/esbbq/esbbq_spanish_region.yaml' 2026-05-11T14:30:08,667 adding 'lm_eval/tasks/esbbq/utils.py' 2026-05-11T14:30:08,670 adding 'lm_eval/tasks/eus_exams/README.md' 2026-05-11T14:30:08,671 adding 'lm_eval/tasks/eus_exams/configs.py' 2026-05-11T14:30:08,673 adding 'lm_eval/tasks/eus_exams/eus_exams' 2026-05-11T14:30:08,674 adding 'lm_eval/tasks/eus_exams/eus_exams_es' 2026-05-11T14:30:08,675 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml' 2026-05-11T14:30:08,676 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml' 2026-05-11T14:30:08,677 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml' 2026-05-11T14:30:08,678 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml' 2026-05-11T14:30:08,679 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml' 2026-05-11T14:30:08,680 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml' 2026-05-11T14:30:08,681 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml' 2026-05-11T14:30:08,682 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml' 2026-05-11T14:30:08,683 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml' 2026-05-11T14:30:08,684 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml' 2026-05-11T14:30:08,686 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml' 2026-05-11T14:30:08,687 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml' 2026-05-11T14:30:08,688 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml' 2026-05-11T14:30:08,689 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml' 2026-05-11T14:30:08,690 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml' 2026-05-11T14:30:08,691 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml' 2026-05-11T14:30:08,692 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml' 2026-05-11T14:30:08,693 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml' 2026-05-11T14:30:08,694 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml' 2026-05-11T14:30:08,696 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml' 2026-05-11T14:30:08,697 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml' 2026-05-11T14:30:08,698 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml' 2026-05-11T14:30:08,699 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml' 2026-05-11T14:30:08,700 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml' 2026-05-11T14:30:08,701 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml' 2026-05-11T14:30:08,702 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml' 2026-05-11T14:30:08,703 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml' 2026-05-11T14:30:08,705 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml' 2026-05-11T14:30:08,706 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml' 2026-05-11T14:30:08,707 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml' 2026-05-11T14:30:08,708 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml' 2026-05-11T14:30:08,709 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml' 2026-05-11T14:30:08,710 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml' 2026-05-11T14:30:08,711 adding 'lm_eval/tasks/eus_exams/eus_exams_eu' 2026-05-11T14:30:08,713 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml' 2026-05-11T14:30:08,714 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml' 2026-05-11T14:30:08,715 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml' 2026-05-11T14:30:08,716 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml' 2026-05-11T14:30:08,717 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml' 2026-05-11T14:30:08,718 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml' 2026-05-11T14:30:08,719 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml' 2026-05-11T14:30:08,720 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml' 2026-05-11T14:30:08,721 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml' 2026-05-11T14:30:08,722 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml' 2026-05-11T14:30:08,723 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml' 2026-05-11T14:30:08,724 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml' 2026-05-11T14:30:08,725 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml' 2026-05-11T14:30:08,726 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml' 2026-05-11T14:30:08,728 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml' 2026-05-11T14:30:08,729 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml' 2026-05-11T14:30:08,730 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml' 2026-05-11T14:30:08,731 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml' 2026-05-11T14:30:08,732 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml' 2026-05-11T14:30:08,733 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml' 2026-05-11T14:30:08,734 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml' 2026-05-11T14:30:08,735 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml' 2026-05-11T14:30:08,737 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml' 2026-05-11T14:30:08,738 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml' 2026-05-11T14:30:08,739 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml' 2026-05-11T14:30:08,740 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml' 2026-05-11T14:30:08,741 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml' 2026-05-11T14:30:08,742 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml' 2026-05-11T14:30:08,743 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml' 2026-05-11T14:30:08,744 adding 'lm_eval/tasks/eus_exams/utils.py' 2026-05-11T14:30:08,746 adding 'lm_eval/tasks/eus_proficiency/README.md' 2026-05-11T14:30:08,748 adding 'lm_eval/tasks/eus_proficiency/eus_proficiency.yaml' 2026-05-11T14:30:08,750 adding 'lm_eval/tasks/eus_reading/README.md' 2026-05-11T14:30:08,751 adding 'lm_eval/tasks/eus_reading/eus_reading.yaml' 2026-05-11T14:30:08,752 adding 'lm_eval/tasks/eus_reading/utils.py' 2026-05-11T14:30:08,754 adding 'lm_eval/tasks/eus_trivia/README.md' 2026-05-11T14:30:08,755 adding 'lm_eval/tasks/eus_trivia/eus_trivia.yaml' 2026-05-11T14:30:08,756 adding 'lm_eval/tasks/eus_trivia/utils.py' 2026-05-11T14:30:08,760 adding 'lm_eval/tasks/evalita_llm/README.md' 2026-05-11T14:30:08,761 adding 'lm_eval/tasks/evalita_llm/_at_template_yaml' 2026-05-11T14:30:08,762 adding 'lm_eval/tasks/evalita_llm/_evalita-mp.yaml' 2026-05-11T14:30:08,763 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p1.yaml' 2026-05-11T14:30:08,765 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p2.yaml' 2026-05-11T14:30:08,766 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p3.yaml' 2026-05-11T14:30:08,767 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p4.yaml' 2026-05-11T14:30:08,768 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p5.yaml' 2026-05-11T14:30:08,770 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p6.yaml' 2026-05-11T14:30:08,771 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_tasks.yaml' 2026-05-11T14:30:08,772 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p1.yaml' 2026-05-11T14:30:08,773 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p2.yaml' 2026-05-11T14:30:08,774 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p3.yaml' 2026-05-11T14:30:08,776 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p4.yaml' 2026-05-11T14:30:08,777 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p5.yaml' 2026-05-11T14:30:08,778 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p6.yaml' 2026-05-11T14:30:08,779 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_tasks.yaml' 2026-05-11T14:30:08,780 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_gen.yaml' 2026-05-11T14:30:08,781 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p1.yaml' 2026-05-11T14:30:08,782 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p2.yaml' 2026-05-11T14:30:08,783 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p3.yaml' 2026-05-11T14:30:08,784 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p4.yaml' 2026-05-11T14:30:08,786 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p5.yaml' 2026-05-11T14:30:08,787 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p6.yaml' 2026-05-11T14:30:08,788 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_task.yaml' 2026-05-11T14:30:08,789 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ls_p1.yaml' 2026-05-11T14:30:08,790 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ls_p2.yaml' 2026-05-11T14:30:08,791 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ls_task.yaml' 2026-05-11T14:30:08,793 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_mc.yaml' 2026-05-11T14:30:08,794 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group.yaml' 2026-05-11T14:30:08,795 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p1.yaml' 2026-05-11T14:30:08,796 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p2.yaml' 2026-05-11T14:30:08,797 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group.yaml' 2026-05-11T14:30:08,798 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p1.yaml' 2026-05-11T14:30:08,800 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p2.yaml' 2026-05-11T14:30:08,801 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group.yaml' 2026-05-11T14:30:08,802 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p1.yaml' 2026-05-11T14:30:08,803 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p2.yaml' 2026-05-11T14:30:08,804 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg' 2026-05-11T14:30:08,805 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p1.yaml' 2026-05-11T14:30:08,807 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p2.yaml' 2026-05-11T14:30:08,808 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic' 2026-05-11T14:30:08,809 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p1.yaml' 2026-05-11T14:30:08,810 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p2.yaml' 2026-05-11T14:30:08,811 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_group.yaml' 2026-05-11T14:30:08,812 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn' 2026-05-11T14:30:08,814 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p1.yaml' 2026-05-11T14:30:08,815 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p2.yaml' 2026-05-11T14:30:08,816 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_re_p1.yaml' 2026-05-11T14:30:08,817 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_re_p2.yaml' 2026-05-11T14:30:08,818 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_re_task.yaml' 2026-05-11T14:30:08,820 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p1.yaml' 2026-05-11T14:30:08,821 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p2.yaml' 2026-05-11T14:30:08,822 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p3.yaml' 2026-05-11T14:30:08,823 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p4.yaml' 2026-05-11T14:30:08,824 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p5.yaml' 2026-05-11T14:30:08,826 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p6.yaml' 2026-05-11T14:30:08,827 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_tasks.yaml' 2026-05-11T14:30:08,828 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p1.yaml' 2026-05-11T14:30:08,829 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p2.yaml' 2026-05-11T14:30:08,830 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_task.yaml' 2026-05-11T14:30:08,831 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p1.yaml' 2026-05-11T14:30:08,832 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p2.yaml' 2026-05-11T14:30:08,833 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_task.yaml' 2026-05-11T14:30:08,834 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p1.yaml' 2026-05-11T14:30:08,836 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p2.yaml' 2026-05-11T14:30:08,837 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p3.yaml' 2026-05-11T14:30:08,838 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p4.yaml' 2026-05-11T14:30:08,839 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p5.yaml' 2026-05-11T14:30:08,840 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p6.yaml' 2026-05-11T14:30:08,841 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_tasks.yaml' 2026-05-11T14:30:08,842 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p1.yaml' 2026-05-11T14:30:08,844 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p2.yaml' 2026-05-11T14:30:08,845 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p3.yaml' 2026-05-11T14:30:08,846 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p4.yaml' 2026-05-11T14:30:08,847 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p5.yaml' 2026-05-11T14:30:08,849 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p6.yaml' 2026-05-11T14:30:08,850 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_tasks.yaml' 2026-05-11T14:30:08,851 adding 'lm_eval/tasks/evalita_llm/_faq_template_yaml' 2026-05-11T14:30:08,852 adding 'lm_eval/tasks/evalita_llm/_hs_template_yaml' 2026-05-11T14:30:08,853 adding 'lm_eval/tasks/evalita_llm/_ls_template_yaml' 2026-05-11T14:30:08,854 adding 'lm_eval/tasks/evalita_llm/_ner_template_yaml' 2026-05-11T14:30:08,856 adding 'lm_eval/tasks/evalita_llm/_re_template_yaml' 2026-05-11T14:30:08,857 adding 'lm_eval/tasks/evalita_llm/_sa_template_v2_yaml' 2026-05-11T14:30:08,858 adding 'lm_eval/tasks/evalita_llm/_sa_template_yaml' 2026-05-11T14:30:08,859 adding 'lm_eval/tasks/evalita_llm/_sum_template_fp-small_yaml' 2026-05-11T14:30:08,860 adding 'lm_eval/tasks/evalita_llm/_sum_template_fp_yaml' 2026-05-11T14:30:08,861 adding 'lm_eval/tasks/evalita_llm/_sum_template_yaml' 2026-05-11T14:30:08,863 adding 'lm_eval/tasks/evalita_llm/_te_template_yaml' 2026-05-11T14:30:08,864 adding 'lm_eval/tasks/evalita_llm/_wic_template_yaml' 2026-05-11T14:30:08,866 adding 'lm_eval/tasks/evalita_llm/metrics.py' 2026-05-11T14:30:08,867 adding 'lm_eval/tasks/evalita_llm/sum_utils.py' 2026-05-11T14:30:08,869 adding 'lm_eval/tasks/evalita_llm/utils.py' 2026-05-11T14:30:08,871 adding 'lm_eval/tasks/fda/README.md' 2026-05-11T14:30:08,872 adding 'lm_eval/tasks/fda/fda.yaml' 2026-05-11T14:30:08,874 adding 'lm_eval/tasks/fda/task.py' 2026-05-11T14:30:08,876 adding 'lm_eval/tasks/fld/README.md' 2026-05-11T14:30:08,877 adding 'lm_eval/tasks/fld/fld_default.yaml' 2026-05-11T14:30:08,878 adding 'lm_eval/tasks/fld/fld_logical_formula_default.yaml' 2026-05-11T14:30:08,879 adding 'lm_eval/tasks/fld/fld_logical_formula_star.yaml' 2026-05-11T14:30:08,880 adding 'lm_eval/tasks/fld/fld_star.yaml' 2026-05-11T14:30:08,883 adding 'lm_eval/tasks/french_bench/README.md' 2026-05-11T14:30:08,884 adding 'lm_eval/tasks/french_bench/_default_template_yaml' 2026-05-11T14:30:08,885 adding 'lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml' 2026-05-11T14:30:08,886 adding 'lm_eval/tasks/french_bench/french_bench_boolqa.yaml' 2026-05-11T14:30:08,887 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2.yaml' 2026-05-11T14:30:08,889 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml' 2026-05-11T14:30:08,890 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml' 2026-05-11T14:30:08,891 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml' 2026-05-11T14:30:08,892 adding 'lm_eval/tasks/french_bench/french_bench_grammar.yaml' 2026-05-11T14:30:08,894 adding 'lm_eval/tasks/french_bench/french_bench_hellaswag.yaml' 2026-05-11T14:30:08,895 adding 'lm_eval/tasks/french_bench/french_bench_multifquad.yaml' 2026-05-11T14:30:08,896 adding 'lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml' 2026-05-11T14:30:08,897 adding 'lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml' 2026-05-11T14:30:08,898 adding 'lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml' 2026-05-11T14:30:08,899 adding 'lm_eval/tasks/french_bench/french_bench_reading_comp.yaml' 2026-05-11T14:30:08,900 adding 'lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml' 2026-05-11T14:30:08,901 adding 'lm_eval/tasks/french_bench/french_bench_trivia.yaml' 2026-05-11T14:30:08,903 adding 'lm_eval/tasks/french_bench/french_bench_vocab.yaml' 2026-05-11T14:30:08,904 adding 'lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml' 2026-05-11T14:30:08,905 adding 'lm_eval/tasks/french_bench/french_bench_xnli.yaml' 2026-05-11T14:30:08,906 adding 'lm_eval/tasks/french_bench/preprocess_wikitext.py' 2026-05-11T14:30:08,908 adding 'lm_eval/tasks/french_bench/utils.py' 2026-05-11T14:30:08,910 adding 'lm_eval/tasks/galician_bench/README.md' 2026-05-11T14:30:08,911 adding 'lm_eval/tasks/galician_bench/belebele_glg_Latn.yaml' 2026-05-11T14:30:08,912 adding 'lm_eval/tasks/galician_bench/galcola.yaml' 2026-05-11T14:30:08,913 adding 'lm_eval/tasks/galician_bench/galician_bench.yaml' 2026-05-11T14:30:08,914 adding 'lm_eval/tasks/galician_bench/mgsm_direct_gl.yaml' 2026-05-11T14:30:08,915 adding 'lm_eval/tasks/galician_bench/openbookqa_gl.yaml' 2026-05-11T14:30:08,917 adding 'lm_eval/tasks/galician_bench/parafrases_gl.yaml' 2026-05-11T14:30:08,918 adding 'lm_eval/tasks/galician_bench/paws_gl.yaml' 2026-05-11T14:30:08,919 adding 'lm_eval/tasks/galician_bench/summarization_gl.yaml' 2026-05-11T14:30:08,920 adding 'lm_eval/tasks/galician_bench/truthfulqa_gl_gen.yaml' 2026-05-11T14:30:08,921 adding 'lm_eval/tasks/galician_bench/truthfulqa_gl_mc1.yaml' 2026-05-11T14:30:08,923 adding 'lm_eval/tasks/galician_bench/truthfulqa_gl_mc2.yaml' 2026-05-11T14:30:08,924 adding 'lm_eval/tasks/galician_bench/utils.py' 2026-05-11T14:30:08,926 adding 'lm_eval/tasks/galician_bench/xnli_gl.yaml' 2026-05-11T14:30:08,927 adding 'lm_eval/tasks/galician_bench/xstorycloze_gl.yaml' 2026-05-11T14:30:08,929 adding 'lm_eval/tasks/galician_bench/flores_gl/_flores_common_yaml' 2026-05-11T14:30:08,930 adding 'lm_eval/tasks/galician_bench/flores_gl/create_yamls_flores_gl.py' 2026-05-11T14:30:08,931 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_ca-gl.yaml' 2026-05-11T14:30:08,932 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_de-gl.yaml' 2026-05-11T14:30:08,934 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_en-gl.yaml' 2026-05-11T14:30:08,935 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_es-gl.yaml' 2026-05-11T14:30:08,936 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_eu-gl.yaml' 2026-05-11T14:30:08,937 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_fr-gl.yaml' 2026-05-11T14:30:08,938 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-ca.yaml' 2026-05-11T14:30:08,940 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-de.yaml' 2026-05-11T14:30:08,941 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-en.yaml' 2026-05-11T14:30:08,942 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-es.yaml' 2026-05-11T14:30:08,943 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-eu.yaml' 2026-05-11T14:30:08,944 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-fr.yaml' 2026-05-11T14:30:08,945 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-it.yaml' 2026-05-11T14:30:08,946 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-pt.yaml' 2026-05-11T14:30:08,948 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl.yaml' 2026-05-11T14:30:08,949 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_it-gl.yaml' 2026-05-11T14:30:08,950 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_pt-gl.yaml' 2026-05-11T14:30:08,952 adding 'lm_eval/tasks/glianorex/README.md' 2026-05-11T14:30:08,953 adding 'lm_eval/tasks/glianorex/glianorex.yaml' 2026-05-11T14:30:08,954 adding 'lm_eval/tasks/glianorex/glianorex_en.yaml' 2026-05-11T14:30:08,955 adding 'lm_eval/tasks/glianorex/glianorex_fr.yaml' 2026-05-11T14:30:08,956 adding 'lm_eval/tasks/glianorex/preprocess_glianorex.py' 2026-05-11T14:30:08,958 adding 'lm_eval/tasks/global_mmlu/README.md' 2026-05-11T14:30:08,960 adding 'lm_eval/tasks/global_mmlu/default/ar/_ar_template_yaml' 2026-05-11T14:30:08,961 adding 'lm_eval/tasks/global_mmlu/default/ar/_global_mmlu_ar.yaml' 2026-05-11T14:30:08,962 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_business.yaml' 2026-05-11T14:30:08,963 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_humanities.yaml' 2026-05-11T14:30:08,964 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_medical.yaml' 2026-05-11T14:30:08,965 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_other.yaml' 2026-05-11T14:30:08,967 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_social_sciences.yaml' 2026-05-11T14:30:08,968 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_stem.yaml' 2026-05-11T14:30:08,969 adding 'lm_eval/tasks/global_mmlu/default/ar/utils.py' 2026-05-11T14:30:08,971 adding 'lm_eval/tasks/global_mmlu/default/bn/_bn_template_yaml' 2026-05-11T14:30:08,972 adding 'lm_eval/tasks/global_mmlu/default/bn/_global_mmlu_bn.yaml' 2026-05-11T14:30:08,973 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_business.yaml' 2026-05-11T14:30:08,974 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_humanities.yaml' 2026-05-11T14:30:08,975 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_medical.yaml' 2026-05-11T14:30:08,976 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_other.yaml' 2026-05-11T14:30:08,977 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_social_sciences.yaml' 2026-05-11T14:30:08,978 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_stem.yaml' 2026-05-11T14:30:08,980 adding 'lm_eval/tasks/global_mmlu/default/bn/utils.py' 2026-05-11T14:30:08,981 adding 'lm_eval/tasks/global_mmlu/default/de/_de_template_yaml' 2026-05-11T14:30:08,983 adding 'lm_eval/tasks/global_mmlu/default/de/_global_mmlu_de.yaml' 2026-05-11T14:30:08,984 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_business.yaml' 2026-05-11T14:30:08,985 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_humanities.yaml' 2026-05-11T14:30:08,986 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_medical.yaml' 2026-05-11T14:30:08,987 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_other.yaml' 2026-05-11T14:30:08,989 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_social_sciences.yaml' 2026-05-11T14:30:08,990 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_stem.yaml' 2026-05-11T14:30:08,991 adding 'lm_eval/tasks/global_mmlu/default/de/utils.py' 2026-05-11T14:30:08,993 adding 'lm_eval/tasks/global_mmlu/default/en/_en_template_yaml' 2026-05-11T14:30:08,994 adding 'lm_eval/tasks/global_mmlu/default/en/_global_mmlu_en.yaml' 2026-05-11T14:30:08,995 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_business.yaml' 2026-05-11T14:30:08,996 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_humanities.yaml' 2026-05-11T14:30:08,997 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_medical.yaml' 2026-05-11T14:30:08,998 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_other.yaml' 2026-05-11T14:30:09,000 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_social_sciences.yaml' 2026-05-11T14:30:09,001 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_stem.yaml' 2026-05-11T14:30:09,002 adding 'lm_eval/tasks/global_mmlu/default/en/utils.py' 2026-05-11T14:30:09,004 adding 'lm_eval/tasks/global_mmlu/default/es/_es_template_yaml' 2026-05-11T14:30:09,005 adding 'lm_eval/tasks/global_mmlu/default/es/_global_mmlu_es.yaml' 2026-05-11T14:30:09,006 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_business.yaml' 2026-05-11T14:30:09,007 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_humanities.yaml' 2026-05-11T14:30:09,008 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_medical.yaml' 2026-05-11T14:30:09,009 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_other.yaml' 2026-05-11T14:30:09,010 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_social_sciences.yaml' 2026-05-11T14:30:09,011 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_stem.yaml' 2026-05-11T14:30:09,013 adding 'lm_eval/tasks/global_mmlu/default/es/utils.py' 2026-05-11T14:30:09,014 adding 'lm_eval/tasks/global_mmlu/default/fr/_fr_template_yaml' 2026-05-11T14:30:09,015 adding 'lm_eval/tasks/global_mmlu/default/fr/_global_mmlu_fr.yaml' 2026-05-11T14:30:09,017 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_business.yaml' 2026-05-11T14:30:09,018 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_humanities.yaml' 2026-05-11T14:30:09,019 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_medical.yaml' 2026-05-11T14:30:09,020 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_other.yaml' 2026-05-11T14:30:09,021 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_social_sciences.yaml' 2026-05-11T14:30:09,022 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_stem.yaml' 2026-05-11T14:30:09,023 adding 'lm_eval/tasks/global_mmlu/default/fr/utils.py' 2026-05-11T14:30:09,025 adding 'lm_eval/tasks/global_mmlu/default/hi/_global_mmlu_hi.yaml' 2026-05-11T14:30:09,026 adding 'lm_eval/tasks/global_mmlu/default/hi/_hi_template_yaml' 2026-05-11T14:30:09,027 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_business.yaml' 2026-05-11T14:30:09,028 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_humanities.yaml' 2026-05-11T14:30:09,029 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_medical.yaml' 2026-05-11T14:30:09,031 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_other.yaml' 2026-05-11T14:30:09,032 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_social_sciences.yaml' 2026-05-11T14:30:09,033 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_stem.yaml' 2026-05-11T14:30:09,034 adding 'lm_eval/tasks/global_mmlu/default/hi/utils.py' 2026-05-11T14:30:09,036 adding 'lm_eval/tasks/global_mmlu/default/id/_global_mmlu_id.yaml' 2026-05-11T14:30:09,037 adding 'lm_eval/tasks/global_mmlu/default/id/_id_template_yaml' 2026-05-11T14:30:09,038 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_business.yaml' 2026-05-11T14:30:09,039 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_humanities.yaml' 2026-05-11T14:30:09,040 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_medical.yaml' 2026-05-11T14:30:09,041 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_other.yaml' 2026-05-11T14:30:09,043 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_social_sciences.yaml' 2026-05-11T14:30:09,044 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_stem.yaml' 2026-05-11T14:30:09,045 adding 'lm_eval/tasks/global_mmlu/default/id/utils.py' 2026-05-11T14:30:09,047 adding 'lm_eval/tasks/global_mmlu/default/it/_global_mmlu_it.yaml' 2026-05-11T14:30:09,048 adding 'lm_eval/tasks/global_mmlu/default/it/_it_template_yaml' 2026-05-11T14:30:09,049 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_business.yaml' 2026-05-11T14:30:09,050 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_humanities.yaml' 2026-05-11T14:30:09,051 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_medical.yaml' 2026-05-11T14:30:09,053 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_other.yaml' 2026-05-11T14:30:09,054 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_social_sciences.yaml' 2026-05-11T14:30:09,055 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_stem.yaml' 2026-05-11T14:30:09,056 adding 'lm_eval/tasks/global_mmlu/default/it/utils.py' 2026-05-11T14:30:09,058 adding 'lm_eval/tasks/global_mmlu/default/ja/_global_mmlu_ja.yaml' 2026-05-11T14:30:09,059 adding 'lm_eval/tasks/global_mmlu/default/ja/_ja_template_yaml' 2026-05-11T14:30:09,060 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_business.yaml' 2026-05-11T14:30:09,061 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml' 2026-05-11T14:30:09,062 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml' 2026-05-11T14:30:09,063 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml' 2026-05-11T14:30:09,064 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml' 2026-05-11T14:30:09,065 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml' 2026-05-11T14:30:09,066 adding 'lm_eval/tasks/global_mmlu/default/ja/utils.py' 2026-05-11T14:30:09,068 adding 'lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml' 2026-05-11T14:30:09,069 adding 'lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml' 2026-05-11T14:30:09,070 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml' 2026-05-11T14:30:09,071 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml' 2026-05-11T14:30:09,072 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml' 2026-05-11T14:30:09,073 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml' 2026-05-11T14:30:09,074 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml' 2026-05-11T14:30:09,076 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml' 2026-05-11T14:30:09,077 adding 'lm_eval/tasks/global_mmlu/default/ko/utils.py' 2026-05-11T14:30:09,079 adding 'lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml' 2026-05-11T14:30:09,080 adding 'lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml' 2026-05-11T14:30:09,081 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml' 2026-05-11T14:30:09,082 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml' 2026-05-11T14:30:09,083 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml' 2026-05-11T14:30:09,085 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_other.yaml' 2026-05-11T14:30:09,086 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_social_sciences.yaml' 2026-05-11T14:30:09,087 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_stem.yaml' 2026-05-11T14:30:09,088 adding 'lm_eval/tasks/global_mmlu/default/pt/utils.py' 2026-05-11T14:30:09,090 adding 'lm_eval/tasks/global_mmlu/default/sw/_global_mmlu_sw.yaml' 2026-05-11T14:30:09,091 adding 'lm_eval/tasks/global_mmlu/default/sw/_sw_template_yaml' 2026-05-11T14:30:09,092 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_business.yaml' 2026-05-11T14:30:09,093 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_humanities.yaml' 2026-05-11T14:30:09,095 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_medical.yaml' 2026-05-11T14:30:09,096 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_other.yaml' 2026-05-11T14:30:09,097 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_social_sciences.yaml' 2026-05-11T14:30:09,098 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_stem.yaml' 2026-05-11T14:30:09,099 adding 'lm_eval/tasks/global_mmlu/default/sw/utils.py' 2026-05-11T14:30:09,101 adding 'lm_eval/tasks/global_mmlu/default/yo/_global_mmlu_yo.yaml' 2026-05-11T14:30:09,102 adding 'lm_eval/tasks/global_mmlu/default/yo/_yo_template_yaml' 2026-05-11T14:30:09,103 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_business.yaml' 2026-05-11T14:30:09,105 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_humanities.yaml' 2026-05-11T14:30:09,106 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_medical.yaml' 2026-05-11T14:30:09,107 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_other.yaml' 2026-05-11T14:30:09,108 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_social_sciences.yaml' 2026-05-11T14:30:09,109 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml' 2026-05-11T14:30:09,110 adding 'lm_eval/tasks/global_mmlu/default/yo/utils.py' 2026-05-11T14:30:09,112 adding 'lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml' 2026-05-11T14:30:09,113 adding 'lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml' 2026-05-11T14:30:09,114 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml' 2026-05-11T14:30:09,115 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml' 2026-05-11T14:30:09,116 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml' 2026-05-11T14:30:09,117 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml' 2026-05-11T14:30:09,118 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml' 2026-05-11T14:30:09,119 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml' 2026-05-11T14:30:09,121 adding 'lm_eval/tasks/global_mmlu/default/zh/utils.py' 2026-05-11T14:30:09,124 adding 'lm_eval/tasks/global_mmlu/full/am/_am_template_yaml' 2026-05-11T14:30:09,125 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml' 2026-05-11T14:30:09,127 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_humanities.yaml' 2026-05-11T14:30:09,128 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_other.yaml' 2026-05-11T14:30:09,129 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_social_sciences.yaml' 2026-05-11T14:30:09,130 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_stem.yaml' 2026-05-11T14:30:09,131 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_abstract_algebra.yaml' 2026-05-11T14:30:09,132 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_anatomy.yaml' 2026-05-11T14:30:09,134 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_astronomy.yaml' 2026-05-11T14:30:09,135 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_business_ethics.yaml' 2026-05-11T14:30:09,136 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_clinical_knowledge.yaml' 2026-05-11T14:30:09,137 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_biology.yaml' 2026-05-11T14:30:09,138 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_chemistry.yaml' 2026-05-11T14:30:09,139 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_computer_science.yaml' 2026-05-11T14:30:09,140 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_mathematics.yaml' 2026-05-11T14:30:09,142 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_medicine.yaml' 2026-05-11T14:30:09,143 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_physics.yaml' 2026-05-11T14:30:09,144 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_computer_security.yaml' 2026-05-11T14:30:09,145 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_conceptual_physics.yaml' 2026-05-11T14:30:09,146 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_econometrics.yaml' 2026-05-11T14:30:09,147 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_electrical_engineering.yaml' 2026-05-11T14:30:09,149 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_elementary_mathematics.yaml' 2026-05-11T14:30:09,150 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_formal_logic.yaml' 2026-05-11T14:30:09,151 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_global_facts.yaml' 2026-05-11T14:30:09,152 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_biology.yaml' 2026-05-11T14:30:09,153 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_chemistry.yaml' 2026-05-11T14:30:09,154 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_computer_science.yaml' 2026-05-11T14:30:09,156 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_european_history.yaml' 2026-05-11T14:30:09,157 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_geography.yaml' 2026-05-11T14:30:09,158 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,159 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,160 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_mathematics.yaml' 2026-05-11T14:30:09,161 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_microeconomics.yaml' 2026-05-11T14:30:09,162 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_physics.yaml' 2026-05-11T14:30:09,163 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_psychology.yaml' 2026-05-11T14:30:09,164 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_statistics.yaml' 2026-05-11T14:30:09,165 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_us_history.yaml' 2026-05-11T14:30:09,166 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_world_history.yaml' 2026-05-11T14:30:09,167 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_aging.yaml' 2026-05-11T14:30:09,169 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_sexuality.yaml' 2026-05-11T14:30:09,170 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_international_law.yaml' 2026-05-11T14:30:09,171 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_jurisprudence.yaml' 2026-05-11T14:30:09,172 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_logical_fallacies.yaml' 2026-05-11T14:30:09,173 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_machine_learning.yaml' 2026-05-11T14:30:09,174 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_management.yaml' 2026-05-11T14:30:09,175 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_marketing.yaml' 2026-05-11T14:30:09,177 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_medical_genetics.yaml' 2026-05-11T14:30:09,178 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_miscellaneous.yaml' 2026-05-11T14:30:09,179 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_disputes.yaml' 2026-05-11T14:30:09,180 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_scenarios.yaml' 2026-05-11T14:30:09,181 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_nutrition.yaml' 2026-05-11T14:30:09,182 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_philosophy.yaml' 2026-05-11T14:30:09,183 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_prehistory.yaml' 2026-05-11T14:30:09,185 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_accounting.yaml' 2026-05-11T14:30:09,186 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_law.yaml' 2026-05-11T14:30:09,187 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_medicine.yaml' 2026-05-11T14:30:09,189 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_psychology.yaml' 2026-05-11T14:30:09,191 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_public_relations.yaml' 2026-05-11T14:30:09,192 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_security_studies.yaml' 2026-05-11T14:30:09,193 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_sociology.yaml' 2026-05-11T14:30:09,194 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_us_foreign_policy.yaml' 2026-05-11T14:30:09,195 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_virology.yaml' 2026-05-11T14:30:09,196 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_world_religions.yaml' 2026-05-11T14:30:09,198 adding 'lm_eval/tasks/global_mmlu/full/am/utils.py' 2026-05-11T14:30:09,200 adding 'lm_eval/tasks/global_mmlu/full/ar/_ar_template_yaml' 2026-05-11T14:30:09,201 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml' 2026-05-11T14:30:09,202 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_humanities.yaml' 2026-05-11T14:30:09,203 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_other.yaml' 2026-05-11T14:30:09,204 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_social_sciences.yaml' 2026-05-11T14:30:09,205 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_stem.yaml' 2026-05-11T14:30:09,207 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_abstract_algebra.yaml' 2026-05-11T14:30:09,208 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_anatomy.yaml' 2026-05-11T14:30:09,209 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_astronomy.yaml' 2026-05-11T14:30:09,210 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_business_ethics.yaml' 2026-05-11T14:30:09,211 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_clinical_knowledge.yaml' 2026-05-11T14:30:09,212 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_biology.yaml' 2026-05-11T14:30:09,213 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_chemistry.yaml' 2026-05-11T14:30:09,214 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_computer_science.yaml' 2026-05-11T14:30:09,216 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_mathematics.yaml' 2026-05-11T14:30:09,217 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_medicine.yaml' 2026-05-11T14:30:09,218 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_physics.yaml' 2026-05-11T14:30:09,219 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_computer_security.yaml' 2026-05-11T14:30:09,220 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_conceptual_physics.yaml' 2026-05-11T14:30:09,221 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_econometrics.yaml' 2026-05-11T14:30:09,222 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_electrical_engineering.yaml' 2026-05-11T14:30:09,224 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_elementary_mathematics.yaml' 2026-05-11T14:30:09,225 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_formal_logic.yaml' 2026-05-11T14:30:09,226 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_global_facts.yaml' 2026-05-11T14:30:09,227 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_biology.yaml' 2026-05-11T14:30:09,228 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_chemistry.yaml' 2026-05-11T14:30:09,229 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_computer_science.yaml' 2026-05-11T14:30:09,231 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_european_history.yaml' 2026-05-11T14:30:09,232 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_geography.yaml' 2026-05-11T14:30:09,233 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,234 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,235 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_mathematics.yaml' 2026-05-11T14:30:09,236 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_microeconomics.yaml' 2026-05-11T14:30:09,238 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_physics.yaml' 2026-05-11T14:30:09,239 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_psychology.yaml' 2026-05-11T14:30:09,240 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_statistics.yaml' 2026-05-11T14:30:09,241 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_us_history.yaml' 2026-05-11T14:30:09,242 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_world_history.yaml' 2026-05-11T14:30:09,243 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_aging.yaml' 2026-05-11T14:30:09,244 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_sexuality.yaml' 2026-05-11T14:30:09,245 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_international_law.yaml' 2026-05-11T14:30:09,246 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_jurisprudence.yaml' 2026-05-11T14:30:09,247 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_logical_fallacies.yaml' 2026-05-11T14:30:09,248 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_machine_learning.yaml' 2026-05-11T14:30:09,249 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_management.yaml' 2026-05-11T14:30:09,251 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_marketing.yaml' 2026-05-11T14:30:09,252 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_medical_genetics.yaml' 2026-05-11T14:30:09,253 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_miscellaneous.yaml' 2026-05-11T14:30:09,254 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_disputes.yaml' 2026-05-11T14:30:09,255 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_scenarios.yaml' 2026-05-11T14:30:09,256 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_nutrition.yaml' 2026-05-11T14:30:09,257 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_philosophy.yaml' 2026-05-11T14:30:09,258 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_prehistory.yaml' 2026-05-11T14:30:09,259 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_accounting.yaml' 2026-05-11T14:30:09,261 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_law.yaml' 2026-05-11T14:30:09,262 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_medicine.yaml' 2026-05-11T14:30:09,263 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_psychology.yaml' 2026-05-11T14:30:09,264 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_public_relations.yaml' 2026-05-11T14:30:09,265 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_security_studies.yaml' 2026-05-11T14:30:09,266 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_sociology.yaml' 2026-05-11T14:30:09,267 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_us_foreign_policy.yaml' 2026-05-11T14:30:09,268 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_virology.yaml' 2026-05-11T14:30:09,270 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_world_religions.yaml' 2026-05-11T14:30:09,271 adding 'lm_eval/tasks/global_mmlu/full/ar/utils.py' 2026-05-11T14:30:09,274 adding 'lm_eval/tasks/global_mmlu/full/bn/_bn_template_yaml' 2026-05-11T14:30:09,275 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml' 2026-05-11T14:30:09,276 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_humanities.yaml' 2026-05-11T14:30:09,278 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_other.yaml' 2026-05-11T14:30:09,279 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_social_sciences.yaml' 2026-05-11T14:30:09,280 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_stem.yaml' 2026-05-11T14:30:09,281 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_abstract_algebra.yaml' 2026-05-11T14:30:09,282 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_anatomy.yaml' 2026-05-11T14:30:09,283 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_astronomy.yaml' 2026-05-11T14:30:09,285 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_business_ethics.yaml' 2026-05-11T14:30:09,286 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_clinical_knowledge.yaml' 2026-05-11T14:30:09,287 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_biology.yaml' 2026-05-11T14:30:09,288 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_chemistry.yaml' 2026-05-11T14:30:09,289 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_computer_science.yaml' 2026-05-11T14:30:09,290 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_mathematics.yaml' 2026-05-11T14:30:09,291 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_medicine.yaml' 2026-05-11T14:30:09,292 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_physics.yaml' 2026-05-11T14:30:09,293 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_computer_security.yaml' 2026-05-11T14:30:09,294 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_conceptual_physics.yaml' 2026-05-11T14:30:09,295 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_econometrics.yaml' 2026-05-11T14:30:09,296 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_electrical_engineering.yaml' 2026-05-11T14:30:09,297 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_elementary_mathematics.yaml' 2026-05-11T14:30:09,299 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_formal_logic.yaml' 2026-05-11T14:30:09,300 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_global_facts.yaml' 2026-05-11T14:30:09,301 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_biology.yaml' 2026-05-11T14:30:09,302 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_chemistry.yaml' 2026-05-11T14:30:09,303 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_computer_science.yaml' 2026-05-11T14:30:09,304 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_european_history.yaml' 2026-05-11T14:30:09,305 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_geography.yaml' 2026-05-11T14:30:09,306 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,307 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,309 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_mathematics.yaml' 2026-05-11T14:30:09,310 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_microeconomics.yaml' 2026-05-11T14:30:09,311 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_physics.yaml' 2026-05-11T14:30:09,312 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_psychology.yaml' 2026-05-11T14:30:09,313 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_statistics.yaml' 2026-05-11T14:30:09,314 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_us_history.yaml' 2026-05-11T14:30:09,316 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_world_history.yaml' 2026-05-11T14:30:09,317 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_aging.yaml' 2026-05-11T14:30:09,318 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_sexuality.yaml' 2026-05-11T14:30:09,319 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_international_law.yaml' 2026-05-11T14:30:09,320 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_jurisprudence.yaml' 2026-05-11T14:30:09,321 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_logical_fallacies.yaml' 2026-05-11T14:30:09,322 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_machine_learning.yaml' 2026-05-11T14:30:09,324 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_management.yaml' 2026-05-11T14:30:09,325 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_marketing.yaml' 2026-05-11T14:30:09,326 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_medical_genetics.yaml' 2026-05-11T14:30:09,327 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_miscellaneous.yaml' 2026-05-11T14:30:09,328 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_disputes.yaml' 2026-05-11T14:30:09,329 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_scenarios.yaml' 2026-05-11T14:30:09,330 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_nutrition.yaml' 2026-05-11T14:30:09,331 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_philosophy.yaml' 2026-05-11T14:30:09,332 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_prehistory.yaml' 2026-05-11T14:30:09,333 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_accounting.yaml' 2026-05-11T14:30:09,334 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_law.yaml' 2026-05-11T14:30:09,336 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_medicine.yaml' 2026-05-11T14:30:09,337 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_psychology.yaml' 2026-05-11T14:30:09,338 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_public_relations.yaml' 2026-05-11T14:30:09,339 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_security_studies.yaml' 2026-05-11T14:30:09,340 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_sociology.yaml' 2026-05-11T14:30:09,341 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_us_foreign_policy.yaml' 2026-05-11T14:30:09,342 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_virology.yaml' 2026-05-11T14:30:09,343 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_world_religions.yaml' 2026-05-11T14:30:09,345 adding 'lm_eval/tasks/global_mmlu/full/bn/utils.py' 2026-05-11T14:30:09,348 adding 'lm_eval/tasks/global_mmlu/full/cs/_cs_template_yaml' 2026-05-11T14:30:09,349 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml' 2026-05-11T14:30:09,350 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_humanities.yaml' 2026-05-11T14:30:09,351 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_other.yaml' 2026-05-11T14:30:09,352 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_social_sciences.yaml' 2026-05-11T14:30:09,354 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_stem.yaml' 2026-05-11T14:30:09,355 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_abstract_algebra.yaml' 2026-05-11T14:30:09,356 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_anatomy.yaml' 2026-05-11T14:30:09,357 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_astronomy.yaml' 2026-05-11T14:30:09,358 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_business_ethics.yaml' 2026-05-11T14:30:09,360 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_clinical_knowledge.yaml' 2026-05-11T14:30:09,361 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_biology.yaml' 2026-05-11T14:30:09,362 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_chemistry.yaml' 2026-05-11T14:30:09,363 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_computer_science.yaml' 2026-05-11T14:30:09,364 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_mathematics.yaml' 2026-05-11T14:30:09,365 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_medicine.yaml' 2026-05-11T14:30:09,367 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_physics.yaml' 2026-05-11T14:30:09,368 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_computer_security.yaml' 2026-05-11T14:30:09,369 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_conceptual_physics.yaml' 2026-05-11T14:30:09,370 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_econometrics.yaml' 2026-05-11T14:30:09,371 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_electrical_engineering.yaml' 2026-05-11T14:30:09,372 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_elementary_mathematics.yaml' 2026-05-11T14:30:09,373 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_formal_logic.yaml' 2026-05-11T14:30:09,374 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_global_facts.yaml' 2026-05-11T14:30:09,376 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_biology.yaml' 2026-05-11T14:30:09,377 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_chemistry.yaml' 2026-05-11T14:30:09,378 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_computer_science.yaml' 2026-05-11T14:30:09,379 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_european_history.yaml' 2026-05-11T14:30:09,380 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_geography.yaml' 2026-05-11T14:30:09,381 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,382 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,384 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_mathematics.yaml' 2026-05-11T14:30:09,385 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_microeconomics.yaml' 2026-05-11T14:30:09,386 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_physics.yaml' 2026-05-11T14:30:09,387 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_psychology.yaml' 2026-05-11T14:30:09,388 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_statistics.yaml' 2026-05-11T14:30:09,389 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_us_history.yaml' 2026-05-11T14:30:09,390 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_world_history.yaml' 2026-05-11T14:30:09,391 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_aging.yaml' 2026-05-11T14:30:09,393 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_sexuality.yaml' 2026-05-11T14:30:09,394 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_international_law.yaml' 2026-05-11T14:30:09,395 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_jurisprudence.yaml' 2026-05-11T14:30:09,396 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_logical_fallacies.yaml' 2026-05-11T14:30:09,397 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_machine_learning.yaml' 2026-05-11T14:30:09,398 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_management.yaml' 2026-05-11T14:30:09,400 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_marketing.yaml' 2026-05-11T14:30:09,401 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_medical_genetics.yaml' 2026-05-11T14:30:09,402 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_miscellaneous.yaml' 2026-05-11T14:30:09,403 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_disputes.yaml' 2026-05-11T14:30:09,404 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_scenarios.yaml' 2026-05-11T14:30:09,405 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_nutrition.yaml' 2026-05-11T14:30:09,407 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_philosophy.yaml' 2026-05-11T14:30:09,408 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_prehistory.yaml' 2026-05-11T14:30:09,409 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_accounting.yaml' 2026-05-11T14:30:09,410 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_law.yaml' 2026-05-11T14:30:09,412 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_medicine.yaml' 2026-05-11T14:30:09,413 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_psychology.yaml' 2026-05-11T14:30:09,414 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_public_relations.yaml' 2026-05-11T14:30:09,415 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_security_studies.yaml' 2026-05-11T14:30:09,416 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_sociology.yaml' 2026-05-11T14:30:09,417 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_us_foreign_policy.yaml' 2026-05-11T14:30:09,419 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_virology.yaml' 2026-05-11T14:30:09,420 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_world_religions.yaml' 2026-05-11T14:30:09,421 adding 'lm_eval/tasks/global_mmlu/full/cs/utils.py' 2026-05-11T14:30:09,424 adding 'lm_eval/tasks/global_mmlu/full/de/_de_template_yaml' 2026-05-11T14:30:09,425 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml' 2026-05-11T14:30:09,426 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_humanities.yaml' 2026-05-11T14:30:09,428 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_other.yaml' 2026-05-11T14:30:09,429 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_social_sciences.yaml' 2026-05-11T14:30:09,430 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_stem.yaml' 2026-05-11T14:30:09,431 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_abstract_algebra.yaml' 2026-05-11T14:30:09,432 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_anatomy.yaml' 2026-05-11T14:30:09,434 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_astronomy.yaml' 2026-05-11T14:30:09,435 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_business_ethics.yaml' 2026-05-11T14:30:09,437 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_clinical_knowledge.yaml' 2026-05-11T14:30:09,438 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_biology.yaml' 2026-05-11T14:30:09,439 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_chemistry.yaml' 2026-05-11T14:30:09,440 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_computer_science.yaml' 2026-05-11T14:30:09,441 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_mathematics.yaml' 2026-05-11T14:30:09,443 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_medicine.yaml' 2026-05-11T14:30:09,444 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_physics.yaml' 2026-05-11T14:30:09,445 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_computer_security.yaml' 2026-05-11T14:30:09,446 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_conceptual_physics.yaml' 2026-05-11T14:30:09,448 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_econometrics.yaml' 2026-05-11T14:30:09,449 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_electrical_engineering.yaml' 2026-05-11T14:30:09,450 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_elementary_mathematics.yaml' 2026-05-11T14:30:09,451 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_formal_logic.yaml' 2026-05-11T14:30:09,452 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_global_facts.yaml' 2026-05-11T14:30:09,454 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_biology.yaml' 2026-05-11T14:30:09,455 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_chemistry.yaml' 2026-05-11T14:30:09,456 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_computer_science.yaml' 2026-05-11T14:30:09,457 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_european_history.yaml' 2026-05-11T14:30:09,458 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_geography.yaml' 2026-05-11T14:30:09,459 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,460 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,462 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_mathematics.yaml' 2026-05-11T14:30:09,463 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_microeconomics.yaml' 2026-05-11T14:30:09,464 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_physics.yaml' 2026-05-11T14:30:09,465 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_psychology.yaml' 2026-05-11T14:30:09,466 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_statistics.yaml' 2026-05-11T14:30:09,467 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_us_history.yaml' 2026-05-11T14:30:09,468 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_world_history.yaml' 2026-05-11T14:30:09,470 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_aging.yaml' 2026-05-11T14:30:09,471 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_sexuality.yaml' 2026-05-11T14:30:09,472 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_international_law.yaml' 2026-05-11T14:30:09,473 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_jurisprudence.yaml' 2026-05-11T14:30:09,475 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_logical_fallacies.yaml' 2026-05-11T14:30:09,476 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_machine_learning.yaml' 2026-05-11T14:30:09,477 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_management.yaml' 2026-05-11T14:30:09,478 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_marketing.yaml' 2026-05-11T14:30:09,479 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_medical_genetics.yaml' 2026-05-11T14:30:09,480 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_miscellaneous.yaml' 2026-05-11T14:30:09,482 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_disputes.yaml' 2026-05-11T14:30:09,483 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_scenarios.yaml' 2026-05-11T14:30:09,484 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_nutrition.yaml' 2026-05-11T14:30:09,485 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_philosophy.yaml' 2026-05-11T14:30:09,486 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_prehistory.yaml' 2026-05-11T14:30:09,487 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_accounting.yaml' 2026-05-11T14:30:09,489 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_law.yaml' 2026-05-11T14:30:09,490 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_medicine.yaml' 2026-05-11T14:30:09,491 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_psychology.yaml' 2026-05-11T14:30:09,492 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_public_relations.yaml' 2026-05-11T14:30:09,493 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_security_studies.yaml' 2026-05-11T14:30:09,495 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_sociology.yaml' 2026-05-11T14:30:09,496 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_us_foreign_policy.yaml' 2026-05-11T14:30:09,497 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_virology.yaml' 2026-05-11T14:30:09,498 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_world_religions.yaml' 2026-05-11T14:30:09,500 adding 'lm_eval/tasks/global_mmlu/full/de/utils.py' 2026-05-11T14:30:09,502 adding 'lm_eval/tasks/global_mmlu/full/el/_el_template_yaml' 2026-05-11T14:30:09,504 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml' 2026-05-11T14:30:09,505 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_humanities.yaml' 2026-05-11T14:30:09,506 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_other.yaml' 2026-05-11T14:30:09,507 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_social_sciences.yaml' 2026-05-11T14:30:09,508 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_stem.yaml' 2026-05-11T14:30:09,509 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_abstract_algebra.yaml' 2026-05-11T14:30:09,510 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_anatomy.yaml' 2026-05-11T14:30:09,512 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_astronomy.yaml' 2026-05-11T14:30:09,513 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_business_ethics.yaml' 2026-05-11T14:30:09,514 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_clinical_knowledge.yaml' 2026-05-11T14:30:09,515 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_biology.yaml' 2026-05-11T14:30:09,516 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_chemistry.yaml' 2026-05-11T14:30:09,517 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_computer_science.yaml' 2026-05-11T14:30:09,519 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_mathematics.yaml' 2026-05-11T14:30:09,520 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_medicine.yaml' 2026-05-11T14:30:09,521 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_physics.yaml' 2026-05-11T14:30:09,522 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_computer_security.yaml' 2026-05-11T14:30:09,523 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_conceptual_physics.yaml' 2026-05-11T14:30:09,524 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_econometrics.yaml' 2026-05-11T14:30:09,525 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_electrical_engineering.yaml' 2026-05-11T14:30:09,527 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_elementary_mathematics.yaml' 2026-05-11T14:30:09,528 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_formal_logic.yaml' 2026-05-11T14:30:09,529 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_global_facts.yaml' 2026-05-11T14:30:09,530 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_biology.yaml' 2026-05-11T14:30:09,531 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_chemistry.yaml' 2026-05-11T14:30:09,532 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_computer_science.yaml' 2026-05-11T14:30:09,534 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_european_history.yaml' 2026-05-11T14:30:09,535 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_geography.yaml' 2026-05-11T14:30:09,536 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,537 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,539 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_mathematics.yaml' 2026-05-11T14:30:09,540 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_microeconomics.yaml' 2026-05-11T14:30:09,541 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_physics.yaml' 2026-05-11T14:30:09,542 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_psychology.yaml' 2026-05-11T14:30:09,543 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_statistics.yaml' 2026-05-11T14:30:09,544 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_us_history.yaml' 2026-05-11T14:30:09,546 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_world_history.yaml' 2026-05-11T14:30:09,547 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_aging.yaml' 2026-05-11T14:30:09,548 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_sexuality.yaml' 2026-05-11T14:30:09,549 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_international_law.yaml' 2026-05-11T14:30:09,550 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_jurisprudence.yaml' 2026-05-11T14:30:09,551 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_logical_fallacies.yaml' 2026-05-11T14:30:09,553 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_machine_learning.yaml' 2026-05-11T14:30:09,554 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_management.yaml' 2026-05-11T14:30:09,555 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_marketing.yaml' 2026-05-11T14:30:09,556 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_medical_genetics.yaml' 2026-05-11T14:30:09,557 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_miscellaneous.yaml' 2026-05-11T14:30:09,558 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_disputes.yaml' 2026-05-11T14:30:09,559 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_scenarios.yaml' 2026-05-11T14:30:09,561 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_nutrition.yaml' 2026-05-11T14:30:09,562 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_philosophy.yaml' 2026-05-11T14:30:09,563 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_prehistory.yaml' 2026-05-11T14:30:09,564 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_accounting.yaml' 2026-05-11T14:30:09,565 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_law.yaml' 2026-05-11T14:30:09,567 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_medicine.yaml' 2026-05-11T14:30:09,568 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_psychology.yaml' 2026-05-11T14:30:09,569 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_public_relations.yaml' 2026-05-11T14:30:09,570 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_security_studies.yaml' 2026-05-11T14:30:09,571 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_sociology.yaml' 2026-05-11T14:30:09,572 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_us_foreign_policy.yaml' 2026-05-11T14:30:09,574 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_virology.yaml' 2026-05-11T14:30:09,575 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_world_religions.yaml' 2026-05-11T14:30:09,576 adding 'lm_eval/tasks/global_mmlu/full/el/utils.py' 2026-05-11T14:30:09,579 adding 'lm_eval/tasks/global_mmlu/full/en/_en_template_yaml' 2026-05-11T14:30:09,580 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml' 2026-05-11T14:30:09,582 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_humanities.yaml' 2026-05-11T14:30:09,583 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_other.yaml' 2026-05-11T14:30:09,584 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_social_sciences.yaml' 2026-05-11T14:30:09,586 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_stem.yaml' 2026-05-11T14:30:09,587 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_abstract_algebra.yaml' 2026-05-11T14:30:09,588 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_anatomy.yaml' 2026-05-11T14:30:09,589 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_astronomy.yaml' 2026-05-11T14:30:09,590 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_business_ethics.yaml' 2026-05-11T14:30:09,591 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_clinical_knowledge.yaml' 2026-05-11T14:30:09,593 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_biology.yaml' 2026-05-11T14:30:09,594 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_chemistry.yaml' 2026-05-11T14:30:09,595 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_computer_science.yaml' 2026-05-11T14:30:09,596 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_mathematics.yaml' 2026-05-11T14:30:09,597 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_medicine.yaml' 2026-05-11T14:30:09,598 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_physics.yaml' 2026-05-11T14:30:09,599 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_computer_security.yaml' 2026-05-11T14:30:09,601 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_conceptual_physics.yaml' 2026-05-11T14:30:09,602 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_econometrics.yaml' 2026-05-11T14:30:09,603 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_electrical_engineering.yaml' 2026-05-11T14:30:09,604 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_elementary_mathematics.yaml' 2026-05-11T14:30:09,605 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_formal_logic.yaml' 2026-05-11T14:30:09,606 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_global_facts.yaml' 2026-05-11T14:30:09,607 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_biology.yaml' 2026-05-11T14:30:09,608 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_chemistry.yaml' 2026-05-11T14:30:09,610 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_computer_science.yaml' 2026-05-11T14:30:09,611 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_european_history.yaml' 2026-05-11T14:30:09,612 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_geography.yaml' 2026-05-11T14:30:09,613 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,614 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,616 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_mathematics.yaml' 2026-05-11T14:30:09,617 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_microeconomics.yaml' 2026-05-11T14:30:09,618 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_physics.yaml' 2026-05-11T14:30:09,619 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_psychology.yaml' 2026-05-11T14:30:09,620 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_statistics.yaml' 2026-05-11T14:30:09,622 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_us_history.yaml' 2026-05-11T14:30:09,623 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_world_history.yaml' 2026-05-11T14:30:09,624 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_aging.yaml' 2026-05-11T14:30:09,625 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_sexuality.yaml' 2026-05-11T14:30:09,626 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_international_law.yaml' 2026-05-11T14:30:09,627 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_jurisprudence.yaml' 2026-05-11T14:30:09,629 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_logical_fallacies.yaml' 2026-05-11T14:30:09,630 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_machine_learning.yaml' 2026-05-11T14:30:09,631 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_management.yaml' 2026-05-11T14:30:09,632 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_marketing.yaml' 2026-05-11T14:30:09,633 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_medical_genetics.yaml' 2026-05-11T14:30:09,635 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_miscellaneous.yaml' 2026-05-11T14:30:09,636 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_disputes.yaml' 2026-05-11T14:30:09,637 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_scenarios.yaml' 2026-05-11T14:30:09,638 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_nutrition.yaml' 2026-05-11T14:30:09,639 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_philosophy.yaml' 2026-05-11T14:30:09,640 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_prehistory.yaml' 2026-05-11T14:30:09,642 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_accounting.yaml' 2026-05-11T14:30:09,643 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_law.yaml' 2026-05-11T14:30:09,644 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_medicine.yaml' 2026-05-11T14:30:09,645 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_psychology.yaml' 2026-05-11T14:30:09,646 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_public_relations.yaml' 2026-05-11T14:30:09,647 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_security_studies.yaml' 2026-05-11T14:30:09,649 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_sociology.yaml' 2026-05-11T14:30:09,650 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_us_foreign_policy.yaml' 2026-05-11T14:30:09,651 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_virology.yaml' 2026-05-11T14:30:09,652 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_world_religions.yaml' 2026-05-11T14:30:09,653 adding 'lm_eval/tasks/global_mmlu/full/en/utils.py' 2026-05-11T14:30:09,656 adding 'lm_eval/tasks/global_mmlu/full/es/_es_template_yaml' 2026-05-11T14:30:09,657 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml' 2026-05-11T14:30:09,659 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_humanities.yaml' 2026-05-11T14:30:09,660 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_other.yaml' 2026-05-11T14:30:09,661 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_social_sciences.yaml' 2026-05-11T14:30:09,662 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_stem.yaml' 2026-05-11T14:30:09,663 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_abstract_algebra.yaml' 2026-05-11T14:30:09,664 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_anatomy.yaml' 2026-05-11T14:30:09,666 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_astronomy.yaml' 2026-05-11T14:30:09,667 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_business_ethics.yaml' 2026-05-11T14:30:09,668 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_clinical_knowledge.yaml' 2026-05-11T14:30:09,669 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_biology.yaml' 2026-05-11T14:30:09,670 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_chemistry.yaml' 2026-05-11T14:30:09,671 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_computer_science.yaml' 2026-05-11T14:30:09,673 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_mathematics.yaml' 2026-05-11T14:30:09,674 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_medicine.yaml' 2026-05-11T14:30:09,675 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_physics.yaml' 2026-05-11T14:30:09,676 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_computer_security.yaml' 2026-05-11T14:30:09,677 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_conceptual_physics.yaml' 2026-05-11T14:30:09,679 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_econometrics.yaml' 2026-05-11T14:30:09,680 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_electrical_engineering.yaml' 2026-05-11T14:30:09,681 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_elementary_mathematics.yaml' 2026-05-11T14:30:09,682 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_formal_logic.yaml' 2026-05-11T14:30:09,683 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_global_facts.yaml' 2026-05-11T14:30:09,684 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_biology.yaml' 2026-05-11T14:30:09,685 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_chemistry.yaml' 2026-05-11T14:30:09,687 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_computer_science.yaml' 2026-05-11T14:30:09,688 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_european_history.yaml' 2026-05-11T14:30:09,689 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_geography.yaml' 2026-05-11T14:30:09,690 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,691 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,692 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_mathematics.yaml' 2026-05-11T14:30:09,694 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_microeconomics.yaml' 2026-05-11T14:30:09,695 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_physics.yaml' 2026-05-11T14:30:09,696 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_psychology.yaml' 2026-05-11T14:30:09,697 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_statistics.yaml' 2026-05-11T14:30:09,699 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_us_history.yaml' 2026-05-11T14:30:09,700 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_world_history.yaml' 2026-05-11T14:30:09,701 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_aging.yaml' 2026-05-11T14:30:09,702 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_sexuality.yaml' 2026-05-11T14:30:09,703 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_international_law.yaml' 2026-05-11T14:30:09,704 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_jurisprudence.yaml' 2026-05-11T14:30:09,706 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_logical_fallacies.yaml' 2026-05-11T14:30:09,707 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_machine_learning.yaml' 2026-05-11T14:30:09,708 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_management.yaml' 2026-05-11T14:30:09,709 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_marketing.yaml' 2026-05-11T14:30:09,710 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_medical_genetics.yaml' 2026-05-11T14:30:09,712 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_miscellaneous.yaml' 2026-05-11T14:30:09,713 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_disputes.yaml' 2026-05-11T14:30:09,714 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_scenarios.yaml' 2026-05-11T14:30:09,715 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_nutrition.yaml' 2026-05-11T14:30:09,716 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_philosophy.yaml' 2026-05-11T14:30:09,718 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_prehistory.yaml' 2026-05-11T14:30:09,719 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_accounting.yaml' 2026-05-11T14:30:09,720 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_law.yaml' 2026-05-11T14:30:09,721 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_medicine.yaml' 2026-05-11T14:30:09,722 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_psychology.yaml' 2026-05-11T14:30:09,724 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_public_relations.yaml' 2026-05-11T14:30:09,725 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_security_studies.yaml' 2026-05-11T14:30:09,726 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_sociology.yaml' 2026-05-11T14:30:09,727 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_us_foreign_policy.yaml' 2026-05-11T14:30:09,728 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_virology.yaml' 2026-05-11T14:30:09,729 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_world_religions.yaml' 2026-05-11T14:30:09,730 adding 'lm_eval/tasks/global_mmlu/full/es/utils.py' 2026-05-11T14:30:09,733 adding 'lm_eval/tasks/global_mmlu/full/fa/_fa_template_yaml' 2026-05-11T14:30:09,734 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml' 2026-05-11T14:30:09,735 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_humanities.yaml' 2026-05-11T14:30:09,736 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_other.yaml' 2026-05-11T14:30:09,738 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_social_sciences.yaml' 2026-05-11T14:30:09,739 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_stem.yaml' 2026-05-11T14:30:09,740 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_abstract_algebra.yaml' 2026-05-11T14:30:09,741 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_anatomy.yaml' 2026-05-11T14:30:09,742 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_astronomy.yaml' 2026-05-11T14:30:09,743 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_business_ethics.yaml' 2026-05-11T14:30:09,744 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_clinical_knowledge.yaml' 2026-05-11T14:30:09,745 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_biology.yaml' 2026-05-11T14:30:09,747 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_chemistry.yaml' 2026-05-11T14:30:09,748 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_computer_science.yaml' 2026-05-11T14:30:09,749 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_mathematics.yaml' 2026-05-11T14:30:09,750 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_medicine.yaml' 2026-05-11T14:30:09,751 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_physics.yaml' 2026-05-11T14:30:09,752 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_computer_security.yaml' 2026-05-11T14:30:09,754 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_conceptual_physics.yaml' 2026-05-11T14:30:09,755 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_econometrics.yaml' 2026-05-11T14:30:09,756 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_electrical_engineering.yaml' 2026-05-11T14:30:09,757 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_elementary_mathematics.yaml' 2026-05-11T14:30:09,758 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_formal_logic.yaml' 2026-05-11T14:30:09,760 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_global_facts.yaml' 2026-05-11T14:30:09,761 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_biology.yaml' 2026-05-11T14:30:09,762 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_chemistry.yaml' 2026-05-11T14:30:09,763 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_computer_science.yaml' 2026-05-11T14:30:09,764 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_european_history.yaml' 2026-05-11T14:30:09,766 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_geography.yaml' 2026-05-11T14:30:09,767 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,768 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,769 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_mathematics.yaml' 2026-05-11T14:30:09,770 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_microeconomics.yaml' 2026-05-11T14:30:09,771 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_physics.yaml' 2026-05-11T14:30:09,773 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_psychology.yaml' 2026-05-11T14:30:09,774 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_statistics.yaml' 2026-05-11T14:30:09,775 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_us_history.yaml' 2026-05-11T14:30:09,776 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_world_history.yaml' 2026-05-11T14:30:09,777 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_aging.yaml' 2026-05-11T14:30:09,778 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_sexuality.yaml' 2026-05-11T14:30:09,779 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_international_law.yaml' 2026-05-11T14:30:09,780 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_jurisprudence.yaml' 2026-05-11T14:30:09,781 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_logical_fallacies.yaml' 2026-05-11T14:30:09,783 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_machine_learning.yaml' 2026-05-11T14:30:09,784 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_management.yaml' 2026-05-11T14:30:09,785 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_marketing.yaml' 2026-05-11T14:30:09,786 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_medical_genetics.yaml' 2026-05-11T14:30:09,787 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_miscellaneous.yaml' 2026-05-11T14:30:09,788 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_disputes.yaml' 2026-05-11T14:30:09,789 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_scenarios.yaml' 2026-05-11T14:30:09,790 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_nutrition.yaml' 2026-05-11T14:30:09,792 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_philosophy.yaml' 2026-05-11T14:30:09,793 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_prehistory.yaml' 2026-05-11T14:30:09,794 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_accounting.yaml' 2026-05-11T14:30:09,795 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_law.yaml' 2026-05-11T14:30:09,796 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_medicine.yaml' 2026-05-11T14:30:09,797 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_psychology.yaml' 2026-05-11T14:30:09,798 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_public_relations.yaml' 2026-05-11T14:30:09,800 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_security_studies.yaml' 2026-05-11T14:30:09,801 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_sociology.yaml' 2026-05-11T14:30:09,802 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_us_foreign_policy.yaml' 2026-05-11T14:30:09,803 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_virology.yaml' 2026-05-11T14:30:09,804 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_world_religions.yaml' 2026-05-11T14:30:09,805 adding 'lm_eval/tasks/global_mmlu/full/fa/utils.py' 2026-05-11T14:30:09,808 adding 'lm_eval/tasks/global_mmlu/full/fil/_fil_template_yaml' 2026-05-11T14:30:09,810 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil.yaml' 2026-05-11T14:30:09,811 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_humanities.yaml' 2026-05-11T14:30:09,812 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_other.yaml' 2026-05-11T14:30:09,813 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_social_sciences.yaml' 2026-05-11T14:30:09,814 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_stem.yaml' 2026-05-11T14:30:09,815 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_abstract_algebra.yaml' 2026-05-11T14:30:09,817 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_anatomy.yaml' 2026-05-11T14:30:09,818 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_astronomy.yaml' 2026-05-11T14:30:09,819 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_business_ethics.yaml' 2026-05-11T14:30:09,820 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_clinical_knowledge.yaml' 2026-05-11T14:30:09,821 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_biology.yaml' 2026-05-11T14:30:09,822 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_chemistry.yaml' 2026-05-11T14:30:09,823 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_computer_science.yaml' 2026-05-11T14:30:09,824 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_mathematics.yaml' 2026-05-11T14:30:09,825 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_medicine.yaml' 2026-05-11T14:30:09,826 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_physics.yaml' 2026-05-11T14:30:09,828 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_computer_security.yaml' 2026-05-11T14:30:09,829 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_conceptual_physics.yaml' 2026-05-11T14:30:09,830 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_econometrics.yaml' 2026-05-11T14:30:09,831 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_electrical_engineering.yaml' 2026-05-11T14:30:09,832 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_elementary_mathematics.yaml' 2026-05-11T14:30:09,833 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_formal_logic.yaml' 2026-05-11T14:30:09,834 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_global_facts.yaml' 2026-05-11T14:30:09,835 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_biology.yaml' 2026-05-11T14:30:09,836 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_chemistry.yaml' 2026-05-11T14:30:09,838 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_computer_science.yaml' 2026-05-11T14:30:09,839 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_european_history.yaml' 2026-05-11T14:30:09,840 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_geography.yaml' 2026-05-11T14:30:09,841 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,842 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,843 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_mathematics.yaml' 2026-05-11T14:30:09,845 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_microeconomics.yaml' 2026-05-11T14:30:09,846 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_physics.yaml' 2026-05-11T14:30:09,847 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_psychology.yaml' 2026-05-11T14:30:09,848 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_statistics.yaml' 2026-05-11T14:30:09,849 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_us_history.yaml' 2026-05-11T14:30:09,850 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_world_history.yaml' 2026-05-11T14:30:09,851 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_aging.yaml' 2026-05-11T14:30:09,853 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_sexuality.yaml' 2026-05-11T14:30:09,854 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_international_law.yaml' 2026-05-11T14:30:09,855 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_jurisprudence.yaml' 2026-05-11T14:30:09,856 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_logical_fallacies.yaml' 2026-05-11T14:30:09,857 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_machine_learning.yaml' 2026-05-11T14:30:09,858 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_management.yaml' 2026-05-11T14:30:09,859 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_marketing.yaml' 2026-05-11T14:30:09,860 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_medical_genetics.yaml' 2026-05-11T14:30:09,861 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_miscellaneous.yaml' 2026-05-11T14:30:09,862 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_disputes.yaml' 2026-05-11T14:30:09,863 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_scenarios.yaml' 2026-05-11T14:30:09,865 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_nutrition.yaml' 2026-05-11T14:30:09,866 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_philosophy.yaml' 2026-05-11T14:30:09,867 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_prehistory.yaml' 2026-05-11T14:30:09,868 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_accounting.yaml' 2026-05-11T14:30:09,869 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_law.yaml' 2026-05-11T14:30:09,870 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_medicine.yaml' 2026-05-11T14:30:09,871 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_psychology.yaml' 2026-05-11T14:30:09,873 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_public_relations.yaml' 2026-05-11T14:30:09,874 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_security_studies.yaml' 2026-05-11T14:30:09,875 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_sociology.yaml' 2026-05-11T14:30:09,876 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_us_foreign_policy.yaml' 2026-05-11T14:30:09,877 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_virology.yaml' 2026-05-11T14:30:09,878 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_world_religions.yaml' 2026-05-11T14:30:09,880 adding 'lm_eval/tasks/global_mmlu/full/fil/utils.py' 2026-05-11T14:30:09,882 adding 'lm_eval/tasks/global_mmlu/full/fr/_fr_template_yaml' 2026-05-11T14:30:09,884 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr.yaml' 2026-05-11T14:30:09,885 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_humanities.yaml' 2026-05-11T14:30:09,886 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_other.yaml' 2026-05-11T14:30:09,887 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_social_sciences.yaml' 2026-05-11T14:30:09,888 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_stem.yaml' 2026-05-11T14:30:09,889 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_abstract_algebra.yaml' 2026-05-11T14:30:09,891 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_anatomy.yaml' 2026-05-11T14:30:09,892 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_astronomy.yaml' 2026-05-11T14:30:09,893 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_business_ethics.yaml' 2026-05-11T14:30:09,894 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_clinical_knowledge.yaml' 2026-05-11T14:30:09,895 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_biology.yaml' 2026-05-11T14:30:09,896 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_chemistry.yaml' 2026-05-11T14:30:09,898 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_computer_science.yaml' 2026-05-11T14:30:09,899 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_mathematics.yaml' 2026-05-11T14:30:09,900 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_medicine.yaml' 2026-05-11T14:30:09,901 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_physics.yaml' 2026-05-11T14:30:09,902 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_computer_security.yaml' 2026-05-11T14:30:09,903 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_conceptual_physics.yaml' 2026-05-11T14:30:09,904 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_econometrics.yaml' 2026-05-11T14:30:09,905 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_electrical_engineering.yaml' 2026-05-11T14:30:09,906 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_elementary_mathematics.yaml' 2026-05-11T14:30:09,907 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_formal_logic.yaml' 2026-05-11T14:30:09,908 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_global_facts.yaml' 2026-05-11T14:30:09,909 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_biology.yaml' 2026-05-11T14:30:09,911 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_chemistry.yaml' 2026-05-11T14:30:09,912 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_computer_science.yaml' 2026-05-11T14:30:09,913 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_european_history.yaml' 2026-05-11T14:30:09,914 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_geography.yaml' 2026-05-11T14:30:09,915 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,916 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,917 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_mathematics.yaml' 2026-05-11T14:30:09,918 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_microeconomics.yaml' 2026-05-11T14:30:09,920 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_physics.yaml' 2026-05-11T14:30:09,921 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_psychology.yaml' 2026-05-11T14:30:09,922 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_statistics.yaml' 2026-05-11T14:30:09,923 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_us_history.yaml' 2026-05-11T14:30:09,924 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_world_history.yaml' 2026-05-11T14:30:09,925 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_aging.yaml' 2026-05-11T14:30:09,926 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_sexuality.yaml' 2026-05-11T14:30:09,928 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_international_law.yaml' 2026-05-11T14:30:09,929 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_jurisprudence.yaml' 2026-05-11T14:30:09,930 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_logical_fallacies.yaml' 2026-05-11T14:30:09,931 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_machine_learning.yaml' 2026-05-11T14:30:09,932 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_management.yaml' 2026-05-11T14:30:09,933 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_marketing.yaml' 2026-05-11T14:30:09,934 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_medical_genetics.yaml' 2026-05-11T14:30:09,936 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_miscellaneous.yaml' 2026-05-11T14:30:09,937 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_disputes.yaml' 2026-05-11T14:30:09,938 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_scenarios.yaml' 2026-05-11T14:30:09,939 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_nutrition.yaml' 2026-05-11T14:30:09,940 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_philosophy.yaml' 2026-05-11T14:30:09,941 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_prehistory.yaml' 2026-05-11T14:30:09,942 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_accounting.yaml' 2026-05-11T14:30:09,943 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_law.yaml' 2026-05-11T14:30:09,944 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_medicine.yaml' 2026-05-11T14:30:09,945 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_psychology.yaml' 2026-05-11T14:30:09,947 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_public_relations.yaml' 2026-05-11T14:30:09,948 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_security_studies.yaml' 2026-05-11T14:30:09,949 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_sociology.yaml' 2026-05-11T14:30:09,950 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_us_foreign_policy.yaml' 2026-05-11T14:30:09,951 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_virology.yaml' 2026-05-11T14:30:09,952 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_world_religions.yaml' 2026-05-11T14:30:09,953 adding 'lm_eval/tasks/global_mmlu/full/fr/utils.py' 2026-05-11T14:30:09,956 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha.yaml' 2026-05-11T14:30:09,957 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_humanities.yaml' 2026-05-11T14:30:09,959 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_other.yaml' 2026-05-11T14:30:09,960 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_social_sciences.yaml' 2026-05-11T14:30:09,961 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_stem.yaml' 2026-05-11T14:30:09,962 adding 'lm_eval/tasks/global_mmlu/full/ha/_ha_template_yaml' 2026-05-11T14:30:09,963 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_abstract_algebra.yaml' 2026-05-11T14:30:09,964 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_anatomy.yaml' 2026-05-11T14:30:09,966 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_astronomy.yaml' 2026-05-11T14:30:09,967 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_business_ethics.yaml' 2026-05-11T14:30:09,968 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_clinical_knowledge.yaml' 2026-05-11T14:30:09,969 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_biology.yaml' 2026-05-11T14:30:09,970 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_chemistry.yaml' 2026-05-11T14:30:09,971 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_computer_science.yaml' 2026-05-11T14:30:09,973 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_mathematics.yaml' 2026-05-11T14:30:09,974 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_medicine.yaml' 2026-05-11T14:30:09,975 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_physics.yaml' 2026-05-11T14:30:09,976 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_computer_security.yaml' 2026-05-11T14:30:09,977 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_conceptual_physics.yaml' 2026-05-11T14:30:09,978 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_econometrics.yaml' 2026-05-11T14:30:09,980 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_electrical_engineering.yaml' 2026-05-11T14:30:09,981 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_elementary_mathematics.yaml' 2026-05-11T14:30:09,982 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_formal_logic.yaml' 2026-05-11T14:30:09,983 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_global_facts.yaml' 2026-05-11T14:30:09,984 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_biology.yaml' 2026-05-11T14:30:09,985 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_chemistry.yaml' 2026-05-11T14:30:09,986 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_computer_science.yaml' 2026-05-11T14:30:09,987 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_european_history.yaml' 2026-05-11T14:30:09,988 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_geography.yaml' 2026-05-11T14:30:09,989 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_government_and_politics.yaml' 2026-05-11T14:30:09,991 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_macroeconomics.yaml' 2026-05-11T14:30:09,992 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_mathematics.yaml' 2026-05-11T14:30:09,993 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_microeconomics.yaml' 2026-05-11T14:30:09,994 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_physics.yaml' 2026-05-11T14:30:09,995 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_psychology.yaml' 2026-05-11T14:30:09,996 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_statistics.yaml' 2026-05-11T14:30:09,997 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_us_history.yaml' 2026-05-11T14:30:09,998 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_world_history.yaml' 2026-05-11T14:30:10,000 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_aging.yaml' 2026-05-11T14:30:10,001 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_sexuality.yaml' 2026-05-11T14:30:10,002 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_international_law.yaml' 2026-05-11T14:30:10,003 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_jurisprudence.yaml' 2026-05-11T14:30:10,004 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_logical_fallacies.yaml' 2026-05-11T14:30:10,005 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_machine_learning.yaml' 2026-05-11T14:30:10,007 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_management.yaml' 2026-05-11T14:30:10,008 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_marketing.yaml' 2026-05-11T14:30:10,009 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_medical_genetics.yaml' 2026-05-11T14:30:10,010 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_miscellaneous.yaml' 2026-05-11T14:30:10,011 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_disputes.yaml' 2026-05-11T14:30:10,012 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_scenarios.yaml' 2026-05-11T14:30:10,013 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_nutrition.yaml' 2026-05-11T14:30:10,015 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_philosophy.yaml' 2026-05-11T14:30:10,016 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_prehistory.yaml' 2026-05-11T14:30:10,017 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_accounting.yaml' 2026-05-11T14:30:10,018 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_law.yaml' 2026-05-11T14:30:10,019 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_medicine.yaml' 2026-05-11T14:30:10,021 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_psychology.yaml' 2026-05-11T14:30:10,022 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_public_relations.yaml' 2026-05-11T14:30:10,023 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_security_studies.yaml' 2026-05-11T14:30:10,024 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_sociology.yaml' 2026-05-11T14:30:10,025 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_us_foreign_policy.yaml' 2026-05-11T14:30:10,027 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_virology.yaml' 2026-05-11T14:30:10,028 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_world_religions.yaml' 2026-05-11T14:30:10,029 adding 'lm_eval/tasks/global_mmlu/full/ha/utils.py' 2026-05-11T14:30:10,032 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he.yaml' 2026-05-11T14:30:10,033 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_humanities.yaml' 2026-05-11T14:30:10,034 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_other.yaml' 2026-05-11T14:30:10,035 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_social_sciences.yaml' 2026-05-11T14:30:10,036 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_stem.yaml' 2026-05-11T14:30:10,037 adding 'lm_eval/tasks/global_mmlu/full/he/_he_template_yaml' 2026-05-11T14:30:10,038 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_abstract_algebra.yaml' 2026-05-11T14:30:10,040 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_anatomy.yaml' 2026-05-11T14:30:10,041 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_astronomy.yaml' 2026-05-11T14:30:10,042 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_business_ethics.yaml' 2026-05-11T14:30:10,043 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_clinical_knowledge.yaml' 2026-05-11T14:30:10,044 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_biology.yaml' 2026-05-11T14:30:10,045 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_chemistry.yaml' 2026-05-11T14:30:10,046 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_computer_science.yaml' 2026-05-11T14:30:10,048 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_mathematics.yaml' 2026-05-11T14:30:10,049 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_medicine.yaml' 2026-05-11T14:30:10,050 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_physics.yaml' 2026-05-11T14:30:10,051 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_computer_security.yaml' 2026-05-11T14:30:10,052 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_conceptual_physics.yaml' 2026-05-11T14:30:10,053 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_econometrics.yaml' 2026-05-11T14:30:10,054 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_electrical_engineering.yaml' 2026-05-11T14:30:10,056 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_elementary_mathematics.yaml' 2026-05-11T14:30:10,057 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_formal_logic.yaml' 2026-05-11T14:30:10,058 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_global_facts.yaml' 2026-05-11T14:30:10,059 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_biology.yaml' 2026-05-11T14:30:10,060 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_chemistry.yaml' 2026-05-11T14:30:10,061 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_computer_science.yaml' 2026-05-11T14:30:10,062 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_european_history.yaml' 2026-05-11T14:30:10,064 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_geography.yaml' 2026-05-11T14:30:10,065 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,066 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,067 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_mathematics.yaml' 2026-05-11T14:30:10,068 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_microeconomics.yaml' 2026-05-11T14:30:10,069 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_physics.yaml' 2026-05-11T14:30:10,071 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_psychology.yaml' 2026-05-11T14:30:10,072 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_statistics.yaml' 2026-05-11T14:30:10,073 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_us_history.yaml' 2026-05-11T14:30:10,074 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_world_history.yaml' 2026-05-11T14:30:10,075 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_aging.yaml' 2026-05-11T14:30:10,076 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_sexuality.yaml' 2026-05-11T14:30:10,077 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_international_law.yaml' 2026-05-11T14:30:10,078 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_jurisprudence.yaml' 2026-05-11T14:30:10,079 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_logical_fallacies.yaml' 2026-05-11T14:30:10,080 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_machine_learning.yaml' 2026-05-11T14:30:10,081 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_management.yaml' 2026-05-11T14:30:10,082 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_marketing.yaml' 2026-05-11T14:30:10,084 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_medical_genetics.yaml' 2026-05-11T14:30:10,085 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_miscellaneous.yaml' 2026-05-11T14:30:10,086 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_disputes.yaml' 2026-05-11T14:30:10,087 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_scenarios.yaml' 2026-05-11T14:30:10,088 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_nutrition.yaml' 2026-05-11T14:30:10,089 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_philosophy.yaml' 2026-05-11T14:30:10,090 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_prehistory.yaml' 2026-05-11T14:30:10,091 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_accounting.yaml' 2026-05-11T14:30:10,092 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_law.yaml' 2026-05-11T14:30:10,094 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_medicine.yaml' 2026-05-11T14:30:10,095 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_psychology.yaml' 2026-05-11T14:30:10,096 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_public_relations.yaml' 2026-05-11T14:30:10,097 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_security_studies.yaml' 2026-05-11T14:30:10,098 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_sociology.yaml' 2026-05-11T14:30:10,099 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_us_foreign_policy.yaml' 2026-05-11T14:30:10,101 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_virology.yaml' 2026-05-11T14:30:10,102 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_world_religions.yaml' 2026-05-11T14:30:10,103 adding 'lm_eval/tasks/global_mmlu/full/he/utils.py' 2026-05-11T14:30:10,106 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi.yaml' 2026-05-11T14:30:10,107 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_humanities.yaml' 2026-05-11T14:30:10,108 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_other.yaml' 2026-05-11T14:30:10,110 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_social_sciences.yaml' 2026-05-11T14:30:10,111 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_stem.yaml' 2026-05-11T14:30:10,112 adding 'lm_eval/tasks/global_mmlu/full/hi/_hi_template_yaml' 2026-05-11T14:30:10,113 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_abstract_algebra.yaml' 2026-05-11T14:30:10,114 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_anatomy.yaml' 2026-05-11T14:30:10,115 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_astronomy.yaml' 2026-05-11T14:30:10,116 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_business_ethics.yaml' 2026-05-11T14:30:10,117 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_clinical_knowledge.yaml' 2026-05-11T14:30:10,118 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_biology.yaml' 2026-05-11T14:30:10,120 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_chemistry.yaml' 2026-05-11T14:30:10,121 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_computer_science.yaml' 2026-05-11T14:30:10,122 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_mathematics.yaml' 2026-05-11T14:30:10,123 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_medicine.yaml' 2026-05-11T14:30:10,124 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_physics.yaml' 2026-05-11T14:30:10,125 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_computer_security.yaml' 2026-05-11T14:30:10,126 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_conceptual_physics.yaml' 2026-05-11T14:30:10,127 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_econometrics.yaml' 2026-05-11T14:30:10,128 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_electrical_engineering.yaml' 2026-05-11T14:30:10,129 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_elementary_mathematics.yaml' 2026-05-11T14:30:10,131 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_formal_logic.yaml' 2026-05-11T14:30:10,132 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_global_facts.yaml' 2026-05-11T14:30:10,133 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_biology.yaml' 2026-05-11T14:30:10,134 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_chemistry.yaml' 2026-05-11T14:30:10,135 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_computer_science.yaml' 2026-05-11T14:30:10,136 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_european_history.yaml' 2026-05-11T14:30:10,137 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_geography.yaml' 2026-05-11T14:30:10,139 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,140 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,141 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_mathematics.yaml' 2026-05-11T14:30:10,142 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_microeconomics.yaml' 2026-05-11T14:30:10,143 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_physics.yaml' 2026-05-11T14:30:10,144 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_psychology.yaml' 2026-05-11T14:30:10,145 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_statistics.yaml' 2026-05-11T14:30:10,147 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_us_history.yaml' 2026-05-11T14:30:10,148 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_world_history.yaml' 2026-05-11T14:30:10,149 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_aging.yaml' 2026-05-11T14:30:10,150 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_sexuality.yaml' 2026-05-11T14:30:10,152 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_international_law.yaml' 2026-05-11T14:30:10,153 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_jurisprudence.yaml' 2026-05-11T14:30:10,154 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_logical_fallacies.yaml' 2026-05-11T14:30:10,155 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_machine_learning.yaml' 2026-05-11T14:30:10,156 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_management.yaml' 2026-05-11T14:30:10,157 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_marketing.yaml' 2026-05-11T14:30:10,158 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_medical_genetics.yaml' 2026-05-11T14:30:10,159 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_miscellaneous.yaml' 2026-05-11T14:30:10,160 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_disputes.yaml' 2026-05-11T14:30:10,161 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_scenarios.yaml' 2026-05-11T14:30:10,162 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_nutrition.yaml' 2026-05-11T14:30:10,164 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_philosophy.yaml' 2026-05-11T14:30:10,165 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_prehistory.yaml' 2026-05-11T14:30:10,166 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_accounting.yaml' 2026-05-11T14:30:10,167 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_law.yaml' 2026-05-11T14:30:10,168 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_medicine.yaml' 2026-05-11T14:30:10,169 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_psychology.yaml' 2026-05-11T14:30:10,170 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_public_relations.yaml' 2026-05-11T14:30:10,171 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_security_studies.yaml' 2026-05-11T14:30:10,172 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_sociology.yaml' 2026-05-11T14:30:10,174 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_us_foreign_policy.yaml' 2026-05-11T14:30:10,175 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_virology.yaml' 2026-05-11T14:30:10,176 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_world_religions.yaml' 2026-05-11T14:30:10,177 adding 'lm_eval/tasks/global_mmlu/full/hi/utils.py' 2026-05-11T14:30:10,180 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id.yaml' 2026-05-11T14:30:10,181 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_humanities.yaml' 2026-05-11T14:30:10,182 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_other.yaml' 2026-05-11T14:30:10,183 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_social_sciences.yaml' 2026-05-11T14:30:10,184 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_stem.yaml' 2026-05-11T14:30:10,186 adding 'lm_eval/tasks/global_mmlu/full/id/_id_template_yaml' 2026-05-11T14:30:10,187 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_abstract_algebra.yaml' 2026-05-11T14:30:10,188 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_anatomy.yaml' 2026-05-11T14:30:10,189 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_astronomy.yaml' 2026-05-11T14:30:10,190 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_business_ethics.yaml' 2026-05-11T14:30:10,191 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_clinical_knowledge.yaml' 2026-05-11T14:30:10,193 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_biology.yaml' 2026-05-11T14:30:10,194 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_chemistry.yaml' 2026-05-11T14:30:10,195 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_computer_science.yaml' 2026-05-11T14:30:10,196 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_mathematics.yaml' 2026-05-11T14:30:10,197 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_medicine.yaml' 2026-05-11T14:30:10,198 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_physics.yaml' 2026-05-11T14:30:10,199 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_computer_security.yaml' 2026-05-11T14:30:10,201 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_conceptual_physics.yaml' 2026-05-11T14:30:10,202 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_econometrics.yaml' 2026-05-11T14:30:10,203 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_electrical_engineering.yaml' 2026-05-11T14:30:10,204 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_elementary_mathematics.yaml' 2026-05-11T14:30:10,205 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_formal_logic.yaml' 2026-05-11T14:30:10,206 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_global_facts.yaml' 2026-05-11T14:30:10,207 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_biology.yaml' 2026-05-11T14:30:10,208 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_chemistry.yaml' 2026-05-11T14:30:10,209 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_computer_science.yaml' 2026-05-11T14:30:10,210 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_european_history.yaml' 2026-05-11T14:30:10,211 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_geography.yaml' 2026-05-11T14:30:10,213 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,214 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,215 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_mathematics.yaml' 2026-05-11T14:30:10,216 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_microeconomics.yaml' 2026-05-11T14:30:10,217 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_physics.yaml' 2026-05-11T14:30:10,218 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_psychology.yaml' 2026-05-11T14:30:10,219 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_statistics.yaml' 2026-05-11T14:30:10,220 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_us_history.yaml' 2026-05-11T14:30:10,221 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_world_history.yaml' 2026-05-11T14:30:10,222 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_aging.yaml' 2026-05-11T14:30:10,224 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_sexuality.yaml' 2026-05-11T14:30:10,225 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_international_law.yaml' 2026-05-11T14:30:10,226 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_jurisprudence.yaml' 2026-05-11T14:30:10,227 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_logical_fallacies.yaml' 2026-05-11T14:30:10,228 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_machine_learning.yaml' 2026-05-11T14:30:10,231 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_management.yaml' 2026-05-11T14:30:10,232 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_marketing.yaml' 2026-05-11T14:30:10,233 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_medical_genetics.yaml' 2026-05-11T14:30:10,235 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_miscellaneous.yaml' 2026-05-11T14:30:10,236 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_disputes.yaml' 2026-05-11T14:30:10,237 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_scenarios.yaml' 2026-05-11T14:30:10,238 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_nutrition.yaml' 2026-05-11T14:30:10,239 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_philosophy.yaml' 2026-05-11T14:30:10,240 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_prehistory.yaml' 2026-05-11T14:30:10,242 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_accounting.yaml' 2026-05-11T14:30:10,243 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_law.yaml' 2026-05-11T14:30:10,244 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_medicine.yaml' 2026-05-11T14:30:10,245 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_psychology.yaml' 2026-05-11T14:30:10,246 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_public_relations.yaml' 2026-05-11T14:30:10,247 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_security_studies.yaml' 2026-05-11T14:30:10,248 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_sociology.yaml' 2026-05-11T14:30:10,249 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_us_foreign_policy.yaml' 2026-05-11T14:30:10,250 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_virology.yaml' 2026-05-11T14:30:10,251 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_world_religions.yaml' 2026-05-11T14:30:10,253 adding 'lm_eval/tasks/global_mmlu/full/id/utils.py' 2026-05-11T14:30:10,255 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig.yaml' 2026-05-11T14:30:10,257 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_humanities.yaml' 2026-05-11T14:30:10,258 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_other.yaml' 2026-05-11T14:30:10,259 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_social_sciences.yaml' 2026-05-11T14:30:10,260 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_stem.yaml' 2026-05-11T14:30:10,261 adding 'lm_eval/tasks/global_mmlu/full/ig/_ig_template_yaml' 2026-05-11T14:30:10,262 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_abstract_algebra.yaml' 2026-05-11T14:30:10,263 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_anatomy.yaml' 2026-05-11T14:30:10,264 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_astronomy.yaml' 2026-05-11T14:30:10,266 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_business_ethics.yaml' 2026-05-11T14:30:10,267 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_clinical_knowledge.yaml' 2026-05-11T14:30:10,268 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_biology.yaml' 2026-05-11T14:30:10,269 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_chemistry.yaml' 2026-05-11T14:30:10,270 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_computer_science.yaml' 2026-05-11T14:30:10,271 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_mathematics.yaml' 2026-05-11T14:30:10,273 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_medicine.yaml' 2026-05-11T14:30:10,274 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_physics.yaml' 2026-05-11T14:30:10,275 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_computer_security.yaml' 2026-05-11T14:30:10,276 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_conceptual_physics.yaml' 2026-05-11T14:30:10,277 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_econometrics.yaml' 2026-05-11T14:30:10,278 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_electrical_engineering.yaml' 2026-05-11T14:30:10,279 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_elementary_mathematics.yaml' 2026-05-11T14:30:10,281 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_formal_logic.yaml' 2026-05-11T14:30:10,282 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_global_facts.yaml' 2026-05-11T14:30:10,283 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_biology.yaml' 2026-05-11T14:30:10,284 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_chemistry.yaml' 2026-05-11T14:30:10,285 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_computer_science.yaml' 2026-05-11T14:30:10,287 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_european_history.yaml' 2026-05-11T14:30:10,288 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_geography.yaml' 2026-05-11T14:30:10,290 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,291 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,292 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_mathematics.yaml' 2026-05-11T14:30:10,293 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_microeconomics.yaml' 2026-05-11T14:30:10,294 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_physics.yaml' 2026-05-11T14:30:10,295 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_psychology.yaml' 2026-05-11T14:30:10,296 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_statistics.yaml' 2026-05-11T14:30:10,298 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_us_history.yaml' 2026-05-11T14:30:10,299 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_world_history.yaml' 2026-05-11T14:30:10,300 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_aging.yaml' 2026-05-11T14:30:10,301 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_sexuality.yaml' 2026-05-11T14:30:10,302 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_international_law.yaml' 2026-05-11T14:30:10,303 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_jurisprudence.yaml' 2026-05-11T14:30:10,304 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_logical_fallacies.yaml' 2026-05-11T14:30:10,305 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_machine_learning.yaml' 2026-05-11T14:30:10,307 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_management.yaml' 2026-05-11T14:30:10,308 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_marketing.yaml' 2026-05-11T14:30:10,309 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_medical_genetics.yaml' 2026-05-11T14:30:10,310 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_miscellaneous.yaml' 2026-05-11T14:30:10,311 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_disputes.yaml' 2026-05-11T14:30:10,312 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_scenarios.yaml' 2026-05-11T14:30:10,313 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_nutrition.yaml' 2026-05-11T14:30:10,315 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_philosophy.yaml' 2026-05-11T14:30:10,316 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_prehistory.yaml' 2026-05-11T14:30:10,317 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_accounting.yaml' 2026-05-11T14:30:10,318 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_law.yaml' 2026-05-11T14:30:10,319 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_medicine.yaml' 2026-05-11T14:30:10,320 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_psychology.yaml' 2026-05-11T14:30:10,321 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_public_relations.yaml' 2026-05-11T14:30:10,323 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_security_studies.yaml' 2026-05-11T14:30:10,324 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_sociology.yaml' 2026-05-11T14:30:10,325 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_us_foreign_policy.yaml' 2026-05-11T14:30:10,326 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_virology.yaml' 2026-05-11T14:30:10,327 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_world_religions.yaml' 2026-05-11T14:30:10,329 adding 'lm_eval/tasks/global_mmlu/full/ig/utils.py' 2026-05-11T14:30:10,331 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it.yaml' 2026-05-11T14:30:10,332 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_humanities.yaml' 2026-05-11T14:30:10,333 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_other.yaml' 2026-05-11T14:30:10,334 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_social_sciences.yaml' 2026-05-11T14:30:10,335 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_stem.yaml' 2026-05-11T14:30:10,337 adding 'lm_eval/tasks/global_mmlu/full/it/_it_template_yaml' 2026-05-11T14:30:10,338 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_abstract_algebra.yaml' 2026-05-11T14:30:10,339 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_anatomy.yaml' 2026-05-11T14:30:10,340 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_astronomy.yaml' 2026-05-11T14:30:10,341 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_business_ethics.yaml' 2026-05-11T14:30:10,342 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_clinical_knowledge.yaml' 2026-05-11T14:30:10,343 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_biology.yaml' 2026-05-11T14:30:10,345 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_chemistry.yaml' 2026-05-11T14:30:10,346 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_computer_science.yaml' 2026-05-11T14:30:10,347 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_mathematics.yaml' 2026-05-11T14:30:10,348 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_medicine.yaml' 2026-05-11T14:30:10,349 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_physics.yaml' 2026-05-11T14:30:10,350 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_computer_security.yaml' 2026-05-11T14:30:10,351 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_conceptual_physics.yaml' 2026-05-11T14:30:10,352 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_econometrics.yaml' 2026-05-11T14:30:10,353 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_electrical_engineering.yaml' 2026-05-11T14:30:10,355 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_elementary_mathematics.yaml' 2026-05-11T14:30:10,356 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_formal_logic.yaml' 2026-05-11T14:30:10,357 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_global_facts.yaml' 2026-05-11T14:30:10,358 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_biology.yaml' 2026-05-11T14:30:10,359 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_chemistry.yaml' 2026-05-11T14:30:10,360 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_computer_science.yaml' 2026-05-11T14:30:10,362 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_european_history.yaml' 2026-05-11T14:30:10,363 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_geography.yaml' 2026-05-11T14:30:10,364 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,365 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,366 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_mathematics.yaml' 2026-05-11T14:30:10,367 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_microeconomics.yaml' 2026-05-11T14:30:10,368 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_physics.yaml' 2026-05-11T14:30:10,370 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_psychology.yaml' 2026-05-11T14:30:10,371 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_statistics.yaml' 2026-05-11T14:30:10,372 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_us_history.yaml' 2026-05-11T14:30:10,374 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_world_history.yaml' 2026-05-11T14:30:10,375 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_aging.yaml' 2026-05-11T14:30:10,376 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_sexuality.yaml' 2026-05-11T14:30:10,377 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_international_law.yaml' 2026-05-11T14:30:10,378 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_jurisprudence.yaml' 2026-05-11T14:30:10,379 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_logical_fallacies.yaml' 2026-05-11T14:30:10,380 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_machine_learning.yaml' 2026-05-11T14:30:10,381 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_management.yaml' 2026-05-11T14:30:10,382 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_marketing.yaml' 2026-05-11T14:30:10,383 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_medical_genetics.yaml' 2026-05-11T14:30:10,384 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_miscellaneous.yaml' 2026-05-11T14:30:10,385 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_disputes.yaml' 2026-05-11T14:30:10,387 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_scenarios.yaml' 2026-05-11T14:30:10,388 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_nutrition.yaml' 2026-05-11T14:30:10,389 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_philosophy.yaml' 2026-05-11T14:30:10,390 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_prehistory.yaml' 2026-05-11T14:30:10,391 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_accounting.yaml' 2026-05-11T14:30:10,392 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_law.yaml' 2026-05-11T14:30:10,393 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_medicine.yaml' 2026-05-11T14:30:10,394 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_psychology.yaml' 2026-05-11T14:30:10,395 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_public_relations.yaml' 2026-05-11T14:30:10,396 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_security_studies.yaml' 2026-05-11T14:30:10,398 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_sociology.yaml' 2026-05-11T14:30:10,399 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_us_foreign_policy.yaml' 2026-05-11T14:30:10,400 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_virology.yaml' 2026-05-11T14:30:10,401 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_world_religions.yaml' 2026-05-11T14:30:10,402 adding 'lm_eval/tasks/global_mmlu/full/it/utils.py' 2026-05-11T14:30:10,405 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja.yaml' 2026-05-11T14:30:10,406 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_humanities.yaml' 2026-05-11T14:30:10,408 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_other.yaml' 2026-05-11T14:30:10,409 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_social_sciences.yaml' 2026-05-11T14:30:10,410 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_stem.yaml' 2026-05-11T14:30:10,411 adding 'lm_eval/tasks/global_mmlu/full/ja/_ja_template_yaml' 2026-05-11T14:30:10,412 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_abstract_algebra.yaml' 2026-05-11T14:30:10,413 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_anatomy.yaml' 2026-05-11T14:30:10,415 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_astronomy.yaml' 2026-05-11T14:30:10,416 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_business_ethics.yaml' 2026-05-11T14:30:10,417 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_clinical_knowledge.yaml' 2026-05-11T14:30:10,418 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_biology.yaml' 2026-05-11T14:30:10,419 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_chemistry.yaml' 2026-05-11T14:30:10,420 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_computer_science.yaml' 2026-05-11T14:30:10,421 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_mathematics.yaml' 2026-05-11T14:30:10,422 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_medicine.yaml' 2026-05-11T14:30:10,423 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_physics.yaml' 2026-05-11T14:30:10,425 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_computer_security.yaml' 2026-05-11T14:30:10,426 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_conceptual_physics.yaml' 2026-05-11T14:30:10,427 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_econometrics.yaml' 2026-05-11T14:30:10,428 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_electrical_engineering.yaml' 2026-05-11T14:30:10,429 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_elementary_mathematics.yaml' 2026-05-11T14:30:10,430 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_formal_logic.yaml' 2026-05-11T14:30:10,431 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_global_facts.yaml' 2026-05-11T14:30:10,432 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_biology.yaml' 2026-05-11T14:30:10,433 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_chemistry.yaml' 2026-05-11T14:30:10,435 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_computer_science.yaml' 2026-05-11T14:30:10,436 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_european_history.yaml' 2026-05-11T14:30:10,437 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_geography.yaml' 2026-05-11T14:30:10,438 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,439 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,440 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_mathematics.yaml' 2026-05-11T14:30:10,441 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_microeconomics.yaml' 2026-05-11T14:30:10,443 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_physics.yaml' 2026-05-11T14:30:10,444 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_psychology.yaml' 2026-05-11T14:30:10,445 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_statistics.yaml' 2026-05-11T14:30:10,446 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_us_history.yaml' 2026-05-11T14:30:10,447 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_world_history.yaml' 2026-05-11T14:30:10,448 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_aging.yaml' 2026-05-11T14:30:10,449 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_sexuality.yaml' 2026-05-11T14:30:10,451 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_international_law.yaml' 2026-05-11T14:30:10,452 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_jurisprudence.yaml' 2026-05-11T14:30:10,453 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_logical_fallacies.yaml' 2026-05-11T14:30:10,455 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_machine_learning.yaml' 2026-05-11T14:30:10,456 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_management.yaml' 2026-05-11T14:30:10,458 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_marketing.yaml' 2026-05-11T14:30:10,459 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_medical_genetics.yaml' 2026-05-11T14:30:10,460 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_miscellaneous.yaml' 2026-05-11T14:30:10,462 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_disputes.yaml' 2026-05-11T14:30:10,463 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_scenarios.yaml' 2026-05-11T14:30:10,464 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_nutrition.yaml' 2026-05-11T14:30:10,465 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_philosophy.yaml' 2026-05-11T14:30:10,466 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_prehistory.yaml' 2026-05-11T14:30:10,467 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_accounting.yaml' 2026-05-11T14:30:10,468 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_law.yaml' 2026-05-11T14:30:10,469 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_medicine.yaml' 2026-05-11T14:30:10,470 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_psychology.yaml' 2026-05-11T14:30:10,471 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_public_relations.yaml' 2026-05-11T14:30:10,473 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_security_studies.yaml' 2026-05-11T14:30:10,474 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_sociology.yaml' 2026-05-11T14:30:10,475 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_us_foreign_policy.yaml' 2026-05-11T14:30:10,476 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_virology.yaml' 2026-05-11T14:30:10,477 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_world_religions.yaml' 2026-05-11T14:30:10,478 adding 'lm_eval/tasks/global_mmlu/full/ja/utils.py' 2026-05-11T14:30:10,481 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko.yaml' 2026-05-11T14:30:10,482 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_humanities.yaml' 2026-05-11T14:30:10,483 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_other.yaml' 2026-05-11T14:30:10,484 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_social_sciences.yaml' 2026-05-11T14:30:10,485 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_stem.yaml' 2026-05-11T14:30:10,487 adding 'lm_eval/tasks/global_mmlu/full/ko/_ko_template_yaml' 2026-05-11T14:30:10,489 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_abstract_algebra.yaml' 2026-05-11T14:30:10,490 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_anatomy.yaml' 2026-05-11T14:30:10,491 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_astronomy.yaml' 2026-05-11T14:30:10,493 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_business_ethics.yaml' 2026-05-11T14:30:10,494 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_clinical_knowledge.yaml' 2026-05-11T14:30:10,495 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_biology.yaml' 2026-05-11T14:30:10,496 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_chemistry.yaml' 2026-05-11T14:30:10,497 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_computer_science.yaml' 2026-05-11T14:30:10,498 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_mathematics.yaml' 2026-05-11T14:30:10,499 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_medicine.yaml' 2026-05-11T14:30:10,501 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_physics.yaml' 2026-05-11T14:30:10,502 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_computer_security.yaml' 2026-05-11T14:30:10,503 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_conceptual_physics.yaml' 2026-05-11T14:30:10,505 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_econometrics.yaml' 2026-05-11T14:30:10,506 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_electrical_engineering.yaml' 2026-05-11T14:30:10,507 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_elementary_mathematics.yaml' 2026-05-11T14:30:10,508 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_formal_logic.yaml' 2026-05-11T14:30:10,509 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_global_facts.yaml' 2026-05-11T14:30:10,510 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_biology.yaml' 2026-05-11T14:30:10,511 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_chemistry.yaml' 2026-05-11T14:30:10,512 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_computer_science.yaml' 2026-05-11T14:30:10,513 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_european_history.yaml' 2026-05-11T14:30:10,514 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_geography.yaml' 2026-05-11T14:30:10,516 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,517 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,518 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_mathematics.yaml' 2026-05-11T14:30:10,519 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_microeconomics.yaml' 2026-05-11T14:30:10,520 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_physics.yaml' 2026-05-11T14:30:10,521 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_psychology.yaml' 2026-05-11T14:30:10,522 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_statistics.yaml' 2026-05-11T14:30:10,523 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_us_history.yaml' 2026-05-11T14:30:10,525 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_world_history.yaml' 2026-05-11T14:30:10,526 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_aging.yaml' 2026-05-11T14:30:10,527 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_sexuality.yaml' 2026-05-11T14:30:10,528 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_international_law.yaml' 2026-05-11T14:30:10,529 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_jurisprudence.yaml' 2026-05-11T14:30:10,530 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_logical_fallacies.yaml' 2026-05-11T14:30:10,531 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_machine_learning.yaml' 2026-05-11T14:30:10,533 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_management.yaml' 2026-05-11T14:30:10,534 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_marketing.yaml' 2026-05-11T14:30:10,535 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_medical_genetics.yaml' 2026-05-11T14:30:10,536 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_miscellaneous.yaml' 2026-05-11T14:30:10,537 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_disputes.yaml' 2026-05-11T14:30:10,538 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_scenarios.yaml' 2026-05-11T14:30:10,539 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_nutrition.yaml' 2026-05-11T14:30:10,541 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_philosophy.yaml' 2026-05-11T14:30:10,542 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_prehistory.yaml' 2026-05-11T14:30:10,543 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_accounting.yaml' 2026-05-11T14:30:10,544 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_law.yaml' 2026-05-11T14:30:10,545 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_medicine.yaml' 2026-05-11T14:30:10,546 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_psychology.yaml' 2026-05-11T14:30:10,548 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_public_relations.yaml' 2026-05-11T14:30:10,549 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_security_studies.yaml' 2026-05-11T14:30:10,550 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_sociology.yaml' 2026-05-11T14:30:10,551 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_us_foreign_policy.yaml' 2026-05-11T14:30:10,552 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_virology.yaml' 2026-05-11T14:30:10,553 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_world_religions.yaml' 2026-05-11T14:30:10,554 adding 'lm_eval/tasks/global_mmlu/full/ko/utils.py' 2026-05-11T14:30:10,557 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky.yaml' 2026-05-11T14:30:10,558 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_humanities.yaml' 2026-05-11T14:30:10,559 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_other.yaml' 2026-05-11T14:30:10,560 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_social_sciences.yaml' 2026-05-11T14:30:10,562 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_stem.yaml' 2026-05-11T14:30:10,563 adding 'lm_eval/tasks/global_mmlu/full/ky/_ky_template_yaml' 2026-05-11T14:30:10,564 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_abstract_algebra.yaml' 2026-05-11T14:30:10,565 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_anatomy.yaml' 2026-05-11T14:30:10,566 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_astronomy.yaml' 2026-05-11T14:30:10,567 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_business_ethics.yaml' 2026-05-11T14:30:10,568 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_clinical_knowledge.yaml' 2026-05-11T14:30:10,570 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_biology.yaml' 2026-05-11T14:30:10,571 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_chemistry.yaml' 2026-05-11T14:30:10,572 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_computer_science.yaml' 2026-05-11T14:30:10,573 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_mathematics.yaml' 2026-05-11T14:30:10,574 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_medicine.yaml' 2026-05-11T14:30:10,575 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_physics.yaml' 2026-05-11T14:30:10,577 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_computer_security.yaml' 2026-05-11T14:30:10,578 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_conceptual_physics.yaml' 2026-05-11T14:30:10,579 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_econometrics.yaml' 2026-05-11T14:30:10,580 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_electrical_engineering.yaml' 2026-05-11T14:30:10,581 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_elementary_mathematics.yaml' 2026-05-11T14:30:10,582 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_formal_logic.yaml' 2026-05-11T14:30:10,583 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_global_facts.yaml' 2026-05-11T14:30:10,585 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_biology.yaml' 2026-05-11T14:30:10,586 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_chemistry.yaml' 2026-05-11T14:30:10,587 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_computer_science.yaml' 2026-05-11T14:30:10,588 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_european_history.yaml' 2026-05-11T14:30:10,589 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_geography.yaml' 2026-05-11T14:30:10,591 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,592 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,593 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_mathematics.yaml' 2026-05-11T14:30:10,594 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_microeconomics.yaml' 2026-05-11T14:30:10,595 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_physics.yaml' 2026-05-11T14:30:10,597 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_psychology.yaml' 2026-05-11T14:30:10,598 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_statistics.yaml' 2026-05-11T14:30:10,599 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_us_history.yaml' 2026-05-11T14:30:10,600 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_world_history.yaml' 2026-05-11T14:30:10,601 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_aging.yaml' 2026-05-11T14:30:10,603 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_sexuality.yaml' 2026-05-11T14:30:10,604 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_international_law.yaml' 2026-05-11T14:30:10,605 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_jurisprudence.yaml' 2026-05-11T14:30:10,606 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_logical_fallacies.yaml' 2026-05-11T14:30:10,607 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_machine_learning.yaml' 2026-05-11T14:30:10,608 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_management.yaml' 2026-05-11T14:30:10,610 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_marketing.yaml' 2026-05-11T14:30:10,611 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_medical_genetics.yaml' 2026-05-11T14:30:10,612 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_miscellaneous.yaml' 2026-05-11T14:30:10,613 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_disputes.yaml' 2026-05-11T14:30:10,614 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_scenarios.yaml' 2026-05-11T14:30:10,615 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_nutrition.yaml' 2026-05-11T14:30:10,617 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_philosophy.yaml' 2026-05-11T14:30:10,618 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_prehistory.yaml' 2026-05-11T14:30:10,619 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_accounting.yaml' 2026-05-11T14:30:10,620 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_law.yaml' 2026-05-11T14:30:10,621 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_medicine.yaml' 2026-05-11T14:30:10,622 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_psychology.yaml' 2026-05-11T14:30:10,623 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_public_relations.yaml' 2026-05-11T14:30:10,625 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_security_studies.yaml' 2026-05-11T14:30:10,626 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_sociology.yaml' 2026-05-11T14:30:10,627 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_us_foreign_policy.yaml' 2026-05-11T14:30:10,628 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_virology.yaml' 2026-05-11T14:30:10,629 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_world_religions.yaml' 2026-05-11T14:30:10,630 adding 'lm_eval/tasks/global_mmlu/full/ky/utils.py' 2026-05-11T14:30:10,633 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt.yaml' 2026-05-11T14:30:10,635 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_humanities.yaml' 2026-05-11T14:30:10,636 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_other.yaml' 2026-05-11T14:30:10,637 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_social_sciences.yaml' 2026-05-11T14:30:10,638 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_stem.yaml' 2026-05-11T14:30:10,639 adding 'lm_eval/tasks/global_mmlu/full/lt/_lt_template_yaml' 2026-05-11T14:30:10,640 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_abstract_algebra.yaml' 2026-05-11T14:30:10,641 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_anatomy.yaml' 2026-05-11T14:30:10,642 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_astronomy.yaml' 2026-05-11T14:30:10,643 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_business_ethics.yaml' 2026-05-11T14:30:10,644 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_clinical_knowledge.yaml' 2026-05-11T14:30:10,645 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_biology.yaml' 2026-05-11T14:30:10,647 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_chemistry.yaml' 2026-05-11T14:30:10,648 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_computer_science.yaml' 2026-05-11T14:30:10,649 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_mathematics.yaml' 2026-05-11T14:30:10,650 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_medicine.yaml' 2026-05-11T14:30:10,651 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_physics.yaml' 2026-05-11T14:30:10,652 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_computer_security.yaml' 2026-05-11T14:30:10,653 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_conceptual_physics.yaml' 2026-05-11T14:30:10,655 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_econometrics.yaml' 2026-05-11T14:30:10,656 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_electrical_engineering.yaml' 2026-05-11T14:30:10,657 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_elementary_mathematics.yaml' 2026-05-11T14:30:10,658 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_formal_logic.yaml' 2026-05-11T14:30:10,659 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_global_facts.yaml' 2026-05-11T14:30:10,660 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_biology.yaml' 2026-05-11T14:30:10,662 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_chemistry.yaml' 2026-05-11T14:30:10,663 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_computer_science.yaml' 2026-05-11T14:30:10,664 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_european_history.yaml' 2026-05-11T14:30:10,665 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_geography.yaml' 2026-05-11T14:30:10,666 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,667 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,668 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_mathematics.yaml' 2026-05-11T14:30:10,670 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_microeconomics.yaml' 2026-05-11T14:30:10,671 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_physics.yaml' 2026-05-11T14:30:10,672 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_psychology.yaml' 2026-05-11T14:30:10,673 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_statistics.yaml' 2026-05-11T14:30:10,674 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_us_history.yaml' 2026-05-11T14:30:10,675 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_world_history.yaml' 2026-05-11T14:30:10,677 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_aging.yaml' 2026-05-11T14:30:10,678 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_sexuality.yaml' 2026-05-11T14:30:10,679 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_international_law.yaml' 2026-05-11T14:30:10,680 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_jurisprudence.yaml' 2026-05-11T14:30:10,681 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_logical_fallacies.yaml' 2026-05-11T14:30:10,682 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_machine_learning.yaml' 2026-05-11T14:30:10,683 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_management.yaml' 2026-05-11T14:30:10,684 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_marketing.yaml' 2026-05-11T14:30:10,686 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_medical_genetics.yaml' 2026-05-11T14:30:10,687 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_miscellaneous.yaml' 2026-05-11T14:30:10,688 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_disputes.yaml' 2026-05-11T14:30:10,689 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_scenarios.yaml' 2026-05-11T14:30:10,690 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_nutrition.yaml' 2026-05-11T14:30:10,691 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_philosophy.yaml' 2026-05-11T14:30:10,693 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_prehistory.yaml' 2026-05-11T14:30:10,694 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_accounting.yaml' 2026-05-11T14:30:10,695 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_law.yaml' 2026-05-11T14:30:10,696 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_medicine.yaml' 2026-05-11T14:30:10,697 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_psychology.yaml' 2026-05-11T14:30:10,698 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_public_relations.yaml' 2026-05-11T14:30:10,699 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_security_studies.yaml' 2026-05-11T14:30:10,700 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_sociology.yaml' 2026-05-11T14:30:10,701 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_us_foreign_policy.yaml' 2026-05-11T14:30:10,703 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_virology.yaml' 2026-05-11T14:30:10,704 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_world_religions.yaml' 2026-05-11T14:30:10,705 adding 'lm_eval/tasks/global_mmlu/full/lt/utils.py' 2026-05-11T14:30:10,708 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg.yaml' 2026-05-11T14:30:10,709 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_humanities.yaml' 2026-05-11T14:30:10,710 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_other.yaml' 2026-05-11T14:30:10,711 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_social_sciences.yaml' 2026-05-11T14:30:10,713 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_stem.yaml' 2026-05-11T14:30:10,714 adding 'lm_eval/tasks/global_mmlu/full/mg/_mg_template_yaml' 2026-05-11T14:30:10,715 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_abstract_algebra.yaml' 2026-05-11T14:30:10,716 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_anatomy.yaml' 2026-05-11T14:30:10,717 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_astronomy.yaml' 2026-05-11T14:30:10,718 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_business_ethics.yaml' 2026-05-11T14:30:10,720 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_clinical_knowledge.yaml' 2026-05-11T14:30:10,721 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_biology.yaml' 2026-05-11T14:30:10,722 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_chemistry.yaml' 2026-05-11T14:30:10,723 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_computer_science.yaml' 2026-05-11T14:30:10,724 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_mathematics.yaml' 2026-05-11T14:30:10,725 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_medicine.yaml' 2026-05-11T14:30:10,727 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_physics.yaml' 2026-05-11T14:30:10,728 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_computer_security.yaml' 2026-05-11T14:30:10,729 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_conceptual_physics.yaml' 2026-05-11T14:30:10,730 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_econometrics.yaml' 2026-05-11T14:30:10,731 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_electrical_engineering.yaml' 2026-05-11T14:30:10,732 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_elementary_mathematics.yaml' 2026-05-11T14:30:10,733 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_formal_logic.yaml' 2026-05-11T14:30:10,734 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_global_facts.yaml' 2026-05-11T14:30:10,735 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_biology.yaml' 2026-05-11T14:30:10,736 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_chemistry.yaml' 2026-05-11T14:30:10,738 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_computer_science.yaml' 2026-05-11T14:30:10,739 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_european_history.yaml' 2026-05-11T14:30:10,740 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_geography.yaml' 2026-05-11T14:30:10,741 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,742 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,743 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_mathematics.yaml' 2026-05-11T14:30:10,744 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_microeconomics.yaml' 2026-05-11T14:30:10,745 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_physics.yaml' 2026-05-11T14:30:10,747 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_psychology.yaml' 2026-05-11T14:30:10,748 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_statistics.yaml' 2026-05-11T14:30:10,749 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_us_history.yaml' 2026-05-11T14:30:10,750 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_world_history.yaml' 2026-05-11T14:30:10,751 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_aging.yaml' 2026-05-11T14:30:10,752 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_sexuality.yaml' 2026-05-11T14:30:10,754 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_international_law.yaml' 2026-05-11T14:30:10,755 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_jurisprudence.yaml' 2026-05-11T14:30:10,756 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_logical_fallacies.yaml' 2026-05-11T14:30:10,757 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_machine_learning.yaml' 2026-05-11T14:30:10,758 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_management.yaml' 2026-05-11T14:30:10,759 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_marketing.yaml' 2026-05-11T14:30:10,761 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_medical_genetics.yaml' 2026-05-11T14:30:10,762 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_miscellaneous.yaml' 2026-05-11T14:30:10,763 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_disputes.yaml' 2026-05-11T14:30:10,764 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_scenarios.yaml' 2026-05-11T14:30:10,765 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_nutrition.yaml' 2026-05-11T14:30:10,767 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_philosophy.yaml' 2026-05-11T14:30:10,768 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_prehistory.yaml' 2026-05-11T14:30:10,769 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_accounting.yaml' 2026-05-11T14:30:10,770 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_law.yaml' 2026-05-11T14:30:10,771 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_medicine.yaml' 2026-05-11T14:30:10,772 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_psychology.yaml' 2026-05-11T14:30:10,773 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_public_relations.yaml' 2026-05-11T14:30:10,774 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_security_studies.yaml' 2026-05-11T14:30:10,775 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_sociology.yaml' 2026-05-11T14:30:10,776 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_us_foreign_policy.yaml' 2026-05-11T14:30:10,777 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_virology.yaml' 2026-05-11T14:30:10,779 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_world_religions.yaml' 2026-05-11T14:30:10,780 adding 'lm_eval/tasks/global_mmlu/full/mg/utils.py' 2026-05-11T14:30:10,782 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms.yaml' 2026-05-11T14:30:10,784 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_humanities.yaml' 2026-05-11T14:30:10,785 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_other.yaml' 2026-05-11T14:30:10,786 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_social_sciences.yaml' 2026-05-11T14:30:10,787 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_stem.yaml' 2026-05-11T14:30:10,788 adding 'lm_eval/tasks/global_mmlu/full/ms/_ms_template_yaml' 2026-05-11T14:30:10,789 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_abstract_algebra.yaml' 2026-05-11T14:30:10,790 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_anatomy.yaml' 2026-05-11T14:30:10,792 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_astronomy.yaml' 2026-05-11T14:30:10,793 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_business_ethics.yaml' 2026-05-11T14:30:10,794 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_clinical_knowledge.yaml' 2026-05-11T14:30:10,795 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_biology.yaml' 2026-05-11T14:30:10,796 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_chemistry.yaml' 2026-05-11T14:30:10,797 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_computer_science.yaml' 2026-05-11T14:30:10,798 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_mathematics.yaml' 2026-05-11T14:30:10,800 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_medicine.yaml' 2026-05-11T14:30:10,801 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_physics.yaml' 2026-05-11T14:30:10,802 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_computer_security.yaml' 2026-05-11T14:30:10,803 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_conceptual_physics.yaml' 2026-05-11T14:30:10,804 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_econometrics.yaml' 2026-05-11T14:30:10,805 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_electrical_engineering.yaml' 2026-05-11T14:30:10,806 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_elementary_mathematics.yaml' 2026-05-11T14:30:10,808 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_formal_logic.yaml' 2026-05-11T14:30:10,809 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_global_facts.yaml' 2026-05-11T14:30:10,810 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_biology.yaml' 2026-05-11T14:30:10,811 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_chemistry.yaml' 2026-05-11T14:30:10,812 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_computer_science.yaml' 2026-05-11T14:30:10,813 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_european_history.yaml' 2026-05-11T14:30:10,814 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_geography.yaml' 2026-05-11T14:30:10,815 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,816 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,817 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_mathematics.yaml' 2026-05-11T14:30:10,818 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_microeconomics.yaml' 2026-05-11T14:30:10,820 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_physics.yaml' 2026-05-11T14:30:10,821 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_psychology.yaml' 2026-05-11T14:30:10,822 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_statistics.yaml' 2026-05-11T14:30:10,823 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_us_history.yaml' 2026-05-11T14:30:10,824 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_world_history.yaml' 2026-05-11T14:30:10,825 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_aging.yaml' 2026-05-11T14:30:10,826 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_sexuality.yaml' 2026-05-11T14:30:10,827 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_international_law.yaml' 2026-05-11T14:30:10,829 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_jurisprudence.yaml' 2026-05-11T14:30:10,830 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_logical_fallacies.yaml' 2026-05-11T14:30:10,831 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_machine_learning.yaml' 2026-05-11T14:30:10,832 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_management.yaml' 2026-05-11T14:30:10,833 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_marketing.yaml' 2026-05-11T14:30:10,834 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_medical_genetics.yaml' 2026-05-11T14:30:10,835 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_miscellaneous.yaml' 2026-05-11T14:30:10,837 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_disputes.yaml' 2026-05-11T14:30:10,838 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_scenarios.yaml' 2026-05-11T14:30:10,839 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_nutrition.yaml' 2026-05-11T14:30:10,840 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_philosophy.yaml' 2026-05-11T14:30:10,841 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_prehistory.yaml' 2026-05-11T14:30:10,842 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_accounting.yaml' 2026-05-11T14:30:10,843 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_law.yaml' 2026-05-11T14:30:10,845 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_medicine.yaml' 2026-05-11T14:30:10,846 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_psychology.yaml' 2026-05-11T14:30:10,847 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_public_relations.yaml' 2026-05-11T14:30:10,848 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_security_studies.yaml' 2026-05-11T14:30:10,849 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_sociology.yaml' 2026-05-11T14:30:10,850 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_us_foreign_policy.yaml' 2026-05-11T14:30:10,852 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_virology.yaml' 2026-05-11T14:30:10,853 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_world_religions.yaml' 2026-05-11T14:30:10,854 adding 'lm_eval/tasks/global_mmlu/full/ms/utils.py' 2026-05-11T14:30:10,856 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne.yaml' 2026-05-11T14:30:10,857 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_humanities.yaml' 2026-05-11T14:30:10,859 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_other.yaml' 2026-05-11T14:30:10,860 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_social_sciences.yaml' 2026-05-11T14:30:10,861 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_stem.yaml' 2026-05-11T14:30:10,862 adding 'lm_eval/tasks/global_mmlu/full/ne/_ne_template_yaml' 2026-05-11T14:30:10,863 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_abstract_algebra.yaml' 2026-05-11T14:30:10,864 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_anatomy.yaml' 2026-05-11T14:30:10,865 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_astronomy.yaml' 2026-05-11T14:30:10,867 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_business_ethics.yaml' 2026-05-11T14:30:10,868 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_clinical_knowledge.yaml' 2026-05-11T14:30:10,869 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_biology.yaml' 2026-05-11T14:30:10,870 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_chemistry.yaml' 2026-05-11T14:30:10,871 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_computer_science.yaml' 2026-05-11T14:30:10,872 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_mathematics.yaml' 2026-05-11T14:30:10,873 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_medicine.yaml' 2026-05-11T14:30:10,874 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_physics.yaml' 2026-05-11T14:30:10,875 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_computer_security.yaml' 2026-05-11T14:30:10,877 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_conceptual_physics.yaml' 2026-05-11T14:30:10,878 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_econometrics.yaml' 2026-05-11T14:30:10,879 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_electrical_engineering.yaml' 2026-05-11T14:30:10,880 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_elementary_mathematics.yaml' 2026-05-11T14:30:10,881 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_formal_logic.yaml' 2026-05-11T14:30:10,882 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_global_facts.yaml' 2026-05-11T14:30:10,883 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_biology.yaml' 2026-05-11T14:30:10,885 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_chemistry.yaml' 2026-05-11T14:30:10,886 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_computer_science.yaml' 2026-05-11T14:30:10,887 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_european_history.yaml' 2026-05-11T14:30:10,888 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_geography.yaml' 2026-05-11T14:30:10,889 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,890 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,892 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_mathematics.yaml' 2026-05-11T14:30:10,893 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_microeconomics.yaml' 2026-05-11T14:30:10,894 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_physics.yaml' 2026-05-11T14:30:10,895 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_psychology.yaml' 2026-05-11T14:30:10,896 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_statistics.yaml' 2026-05-11T14:30:10,897 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_us_history.yaml' 2026-05-11T14:30:10,898 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_world_history.yaml' 2026-05-11T14:30:10,899 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_aging.yaml' 2026-05-11T14:30:10,900 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_sexuality.yaml' 2026-05-11T14:30:10,901 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_international_law.yaml' 2026-05-11T14:30:10,902 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_jurisprudence.yaml' 2026-05-11T14:30:10,904 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_logical_fallacies.yaml' 2026-05-11T14:30:10,905 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_machine_learning.yaml' 2026-05-11T14:30:10,906 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_management.yaml' 2026-05-11T14:30:10,907 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_marketing.yaml' 2026-05-11T14:30:10,908 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_medical_genetics.yaml' 2026-05-11T14:30:10,909 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_miscellaneous.yaml' 2026-05-11T14:30:10,911 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_disputes.yaml' 2026-05-11T14:30:10,912 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_scenarios.yaml' 2026-05-11T14:30:10,913 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_nutrition.yaml' 2026-05-11T14:30:10,914 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_philosophy.yaml' 2026-05-11T14:30:10,915 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_prehistory.yaml' 2026-05-11T14:30:10,916 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_accounting.yaml' 2026-05-11T14:30:10,917 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_law.yaml' 2026-05-11T14:30:10,919 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_medicine.yaml' 2026-05-11T14:30:10,920 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_psychology.yaml' 2026-05-11T14:30:10,921 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_public_relations.yaml' 2026-05-11T14:30:10,922 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_security_studies.yaml' 2026-05-11T14:30:10,923 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_sociology.yaml' 2026-05-11T14:30:10,924 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_us_foreign_policy.yaml' 2026-05-11T14:30:10,925 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_virology.yaml' 2026-05-11T14:30:10,927 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_world_religions.yaml' 2026-05-11T14:30:10,928 adding 'lm_eval/tasks/global_mmlu/full/ne/utils.py' 2026-05-11T14:30:10,931 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl.yaml' 2026-05-11T14:30:10,932 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_humanities.yaml' 2026-05-11T14:30:10,933 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_other.yaml' 2026-05-11T14:30:10,934 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_social_sciences.yaml' 2026-05-11T14:30:10,935 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_stem.yaml' 2026-05-11T14:30:10,937 adding 'lm_eval/tasks/global_mmlu/full/nl/_nl_template_yaml' 2026-05-11T14:30:10,938 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_abstract_algebra.yaml' 2026-05-11T14:30:10,939 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_anatomy.yaml' 2026-05-11T14:30:10,940 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_astronomy.yaml' 2026-05-11T14:30:10,941 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_business_ethics.yaml' 2026-05-11T14:30:10,942 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_clinical_knowledge.yaml' 2026-05-11T14:30:10,943 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_biology.yaml' 2026-05-11T14:30:10,944 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_chemistry.yaml' 2026-05-11T14:30:10,946 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_computer_science.yaml' 2026-05-11T14:30:10,947 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_mathematics.yaml' 2026-05-11T14:30:10,948 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_medicine.yaml' 2026-05-11T14:30:10,949 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_physics.yaml' 2026-05-11T14:30:10,950 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_computer_security.yaml' 2026-05-11T14:30:10,951 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_conceptual_physics.yaml' 2026-05-11T14:30:10,952 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_econometrics.yaml' 2026-05-11T14:30:10,953 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_electrical_engineering.yaml' 2026-05-11T14:30:10,954 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_elementary_mathematics.yaml' 2026-05-11T14:30:10,956 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_formal_logic.yaml' 2026-05-11T14:30:10,957 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_global_facts.yaml' 2026-05-11T14:30:10,958 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_biology.yaml' 2026-05-11T14:30:10,959 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_chemistry.yaml' 2026-05-11T14:30:10,960 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_computer_science.yaml' 2026-05-11T14:30:10,961 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_european_history.yaml' 2026-05-11T14:30:10,963 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_geography.yaml' 2026-05-11T14:30:10,964 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_government_and_politics.yaml' 2026-05-11T14:30:10,965 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_macroeconomics.yaml' 2026-05-11T14:30:10,966 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_mathematics.yaml' 2026-05-11T14:30:10,967 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_microeconomics.yaml' 2026-05-11T14:30:10,968 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_physics.yaml' 2026-05-11T14:30:10,970 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_psychology.yaml' 2026-05-11T14:30:10,971 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_statistics.yaml' 2026-05-11T14:30:10,972 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_us_history.yaml' 2026-05-11T14:30:10,973 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_world_history.yaml' 2026-05-11T14:30:10,974 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_aging.yaml' 2026-05-11T14:30:10,975 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_sexuality.yaml' 2026-05-11T14:30:10,977 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_international_law.yaml' 2026-05-11T14:30:10,978 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_jurisprudence.yaml' 2026-05-11T14:30:10,979 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_logical_fallacies.yaml' 2026-05-11T14:30:10,980 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_machine_learning.yaml' 2026-05-11T14:30:10,981 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_management.yaml' 2026-05-11T14:30:10,982 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_marketing.yaml' 2026-05-11T14:30:10,983 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_medical_genetics.yaml' 2026-05-11T14:30:10,984 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_miscellaneous.yaml' 2026-05-11T14:30:10,986 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_disputes.yaml' 2026-05-11T14:30:10,987 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_scenarios.yaml' 2026-05-11T14:30:10,988 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_nutrition.yaml' 2026-05-11T14:30:10,989 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_philosophy.yaml' 2026-05-11T14:30:10,990 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_prehistory.yaml' 2026-05-11T14:30:10,991 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_accounting.yaml' 2026-05-11T14:30:10,992 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_law.yaml' 2026-05-11T14:30:10,993 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_medicine.yaml' 2026-05-11T14:30:10,994 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_psychology.yaml' 2026-05-11T14:30:10,996 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_public_relations.yaml' 2026-05-11T14:30:10,997 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_security_studies.yaml' 2026-05-11T14:30:10,998 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_sociology.yaml' 2026-05-11T14:30:10,999 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_us_foreign_policy.yaml' 2026-05-11T14:30:11,000 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_virology.yaml' 2026-05-11T14:30:11,001 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_world_religions.yaml' 2026-05-11T14:30:11,003 adding 'lm_eval/tasks/global_mmlu/full/nl/utils.py' 2026-05-11T14:30:11,006 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny.yaml' 2026-05-11T14:30:11,007 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_humanities.yaml' 2026-05-11T14:30:11,008 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_other.yaml' 2026-05-11T14:30:11,009 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_social_sciences.yaml' 2026-05-11T14:30:11,010 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_stem.yaml' 2026-05-11T14:30:11,012 adding 'lm_eval/tasks/global_mmlu/full/ny/_ny_template_yaml' 2026-05-11T14:30:11,013 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_abstract_algebra.yaml' 2026-05-11T14:30:11,014 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_anatomy.yaml' 2026-05-11T14:30:11,015 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_astronomy.yaml' 2026-05-11T14:30:11,016 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_business_ethics.yaml' 2026-05-11T14:30:11,018 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_clinical_knowledge.yaml' 2026-05-11T14:30:11,019 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_biology.yaml' 2026-05-11T14:30:11,020 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_chemistry.yaml' 2026-05-11T14:30:11,021 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_computer_science.yaml' 2026-05-11T14:30:11,023 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_mathematics.yaml' 2026-05-11T14:30:11,024 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_medicine.yaml' 2026-05-11T14:30:11,025 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_physics.yaml' 2026-05-11T14:30:11,026 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_computer_security.yaml' 2026-05-11T14:30:11,028 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_conceptual_physics.yaml' 2026-05-11T14:30:11,029 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_econometrics.yaml' 2026-05-11T14:30:11,030 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_electrical_engineering.yaml' 2026-05-11T14:30:11,031 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_elementary_mathematics.yaml' 2026-05-11T14:30:11,032 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_formal_logic.yaml' 2026-05-11T14:30:11,033 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_global_facts.yaml' 2026-05-11T14:30:11,034 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_biology.yaml' 2026-05-11T14:30:11,035 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_chemistry.yaml' 2026-05-11T14:30:11,037 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_computer_science.yaml' 2026-05-11T14:30:11,038 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_european_history.yaml' 2026-05-11T14:30:11,039 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_geography.yaml' 2026-05-11T14:30:11,040 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,042 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,043 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_mathematics.yaml' 2026-05-11T14:30:11,044 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_microeconomics.yaml' 2026-05-11T14:30:11,045 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_physics.yaml' 2026-05-11T14:30:11,047 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_psychology.yaml' 2026-05-11T14:30:11,048 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_statistics.yaml' 2026-05-11T14:30:11,049 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_us_history.yaml' 2026-05-11T14:30:11,051 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_world_history.yaml' 2026-05-11T14:30:11,052 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_aging.yaml' 2026-05-11T14:30:11,054 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_sexuality.yaml' 2026-05-11T14:30:11,055 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_international_law.yaml' 2026-05-11T14:30:11,056 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_jurisprudence.yaml' 2026-05-11T14:30:11,057 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_logical_fallacies.yaml' 2026-05-11T14:30:11,058 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_machine_learning.yaml' 2026-05-11T14:30:11,060 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_management.yaml' 2026-05-11T14:30:11,061 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_marketing.yaml' 2026-05-11T14:30:11,062 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_medical_genetics.yaml' 2026-05-11T14:30:11,063 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_miscellaneous.yaml' 2026-05-11T14:30:11,065 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_disputes.yaml' 2026-05-11T14:30:11,066 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_scenarios.yaml' 2026-05-11T14:30:11,067 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_nutrition.yaml' 2026-05-11T14:30:11,068 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_philosophy.yaml' 2026-05-11T14:30:11,069 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_prehistory.yaml' 2026-05-11T14:30:11,070 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_accounting.yaml' 2026-05-11T14:30:11,072 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_law.yaml' 2026-05-11T14:30:11,073 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_medicine.yaml' 2026-05-11T14:30:11,074 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_psychology.yaml' 2026-05-11T14:30:11,075 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_public_relations.yaml' 2026-05-11T14:30:11,076 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_security_studies.yaml' 2026-05-11T14:30:11,078 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_sociology.yaml' 2026-05-11T14:30:11,079 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_us_foreign_policy.yaml' 2026-05-11T14:30:11,080 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_virology.yaml' 2026-05-11T14:30:11,081 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_world_religions.yaml' 2026-05-11T14:30:11,083 adding 'lm_eval/tasks/global_mmlu/full/ny/utils.py' 2026-05-11T14:30:11,087 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl.yaml' 2026-05-11T14:30:11,088 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_humanities.yaml' 2026-05-11T14:30:11,090 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_other.yaml' 2026-05-11T14:30:11,091 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_social_sciences.yaml' 2026-05-11T14:30:11,092 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_stem.yaml' 2026-05-11T14:30:11,093 adding 'lm_eval/tasks/global_mmlu/full/pl/_pl_template_yaml' 2026-05-11T14:30:11,094 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_abstract_algebra.yaml' 2026-05-11T14:30:11,095 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_anatomy.yaml' 2026-05-11T14:30:11,097 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_astronomy.yaml' 2026-05-11T14:30:11,098 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_business_ethics.yaml' 2026-05-11T14:30:11,099 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_clinical_knowledge.yaml' 2026-05-11T14:30:11,100 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_biology.yaml' 2026-05-11T14:30:11,101 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_chemistry.yaml' 2026-05-11T14:30:11,102 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_computer_science.yaml' 2026-05-11T14:30:11,104 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_mathematics.yaml' 2026-05-11T14:30:11,105 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_medicine.yaml' 2026-05-11T14:30:11,106 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_physics.yaml' 2026-05-11T14:30:11,107 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_computer_security.yaml' 2026-05-11T14:30:11,108 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_conceptual_physics.yaml' 2026-05-11T14:30:11,109 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_econometrics.yaml' 2026-05-11T14:30:11,111 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_electrical_engineering.yaml' 2026-05-11T14:30:11,112 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_elementary_mathematics.yaml' 2026-05-11T14:30:11,113 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_formal_logic.yaml' 2026-05-11T14:30:11,114 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_global_facts.yaml' 2026-05-11T14:30:11,116 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_biology.yaml' 2026-05-11T14:30:11,117 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_chemistry.yaml' 2026-05-11T14:30:11,119 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_computer_science.yaml' 2026-05-11T14:30:11,121 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_european_history.yaml' 2026-05-11T14:30:11,123 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_geography.yaml' 2026-05-11T14:30:11,124 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,126 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,127 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_mathematics.yaml' 2026-05-11T14:30:11,128 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_microeconomics.yaml' 2026-05-11T14:30:11,130 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_physics.yaml' 2026-05-11T14:30:11,131 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_psychology.yaml' 2026-05-11T14:30:11,132 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_statistics.yaml' 2026-05-11T14:30:11,133 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_us_history.yaml' 2026-05-11T14:30:11,134 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_world_history.yaml' 2026-05-11T14:30:11,135 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_aging.yaml' 2026-05-11T14:30:11,137 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_sexuality.yaml' 2026-05-11T14:30:11,138 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_international_law.yaml' 2026-05-11T14:30:11,139 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_jurisprudence.yaml' 2026-05-11T14:30:11,140 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_logical_fallacies.yaml' 2026-05-11T14:30:11,141 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_machine_learning.yaml' 2026-05-11T14:30:11,142 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_management.yaml' 2026-05-11T14:30:11,144 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_marketing.yaml' 2026-05-11T14:30:11,145 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_medical_genetics.yaml' 2026-05-11T14:30:11,146 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_miscellaneous.yaml' 2026-05-11T14:30:11,147 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_disputes.yaml' 2026-05-11T14:30:11,148 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_scenarios.yaml' 2026-05-11T14:30:11,149 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_nutrition.yaml' 2026-05-11T14:30:11,151 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_philosophy.yaml' 2026-05-11T14:30:11,152 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_prehistory.yaml' 2026-05-11T14:30:11,153 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_accounting.yaml' 2026-05-11T14:30:11,154 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_law.yaml' 2026-05-11T14:30:11,155 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_medicine.yaml' 2026-05-11T14:30:11,156 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_psychology.yaml' 2026-05-11T14:30:11,158 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_public_relations.yaml' 2026-05-11T14:30:11,159 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_security_studies.yaml' 2026-05-11T14:30:11,160 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_sociology.yaml' 2026-05-11T14:30:11,161 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_us_foreign_policy.yaml' 2026-05-11T14:30:11,162 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_virology.yaml' 2026-05-11T14:30:11,164 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_world_religions.yaml' 2026-05-11T14:30:11,165 adding 'lm_eval/tasks/global_mmlu/full/pl/utils.py' 2026-05-11T14:30:11,168 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt.yaml' 2026-05-11T14:30:11,169 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_humanities.yaml' 2026-05-11T14:30:11,170 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_other.yaml' 2026-05-11T14:30:11,171 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_social_sciences.yaml' 2026-05-11T14:30:11,172 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_stem.yaml' 2026-05-11T14:30:11,174 adding 'lm_eval/tasks/global_mmlu/full/pt/_pt_template_yaml' 2026-05-11T14:30:11,175 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_abstract_algebra.yaml' 2026-05-11T14:30:11,176 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_anatomy.yaml' 2026-05-11T14:30:11,177 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_astronomy.yaml' 2026-05-11T14:30:11,178 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_business_ethics.yaml' 2026-05-11T14:30:11,180 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_clinical_knowledge.yaml' 2026-05-11T14:30:11,181 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_biology.yaml' 2026-05-11T14:30:11,182 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_chemistry.yaml' 2026-05-11T14:30:11,183 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_computer_science.yaml' 2026-05-11T14:30:11,185 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_mathematics.yaml' 2026-05-11T14:30:11,186 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_medicine.yaml' 2026-05-11T14:30:11,187 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_physics.yaml' 2026-05-11T14:30:11,188 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_computer_security.yaml' 2026-05-11T14:30:11,189 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_conceptual_physics.yaml' 2026-05-11T14:30:11,190 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_econometrics.yaml' 2026-05-11T14:30:11,192 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_electrical_engineering.yaml' 2026-05-11T14:30:11,193 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_elementary_mathematics.yaml' 2026-05-11T14:30:11,194 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_formal_logic.yaml' 2026-05-11T14:30:11,195 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_global_facts.yaml' 2026-05-11T14:30:11,196 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_biology.yaml' 2026-05-11T14:30:11,197 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_chemistry.yaml' 2026-05-11T14:30:11,199 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_computer_science.yaml' 2026-05-11T14:30:11,200 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_european_history.yaml' 2026-05-11T14:30:11,201 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_geography.yaml' 2026-05-11T14:30:11,202 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,203 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,205 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_mathematics.yaml' 2026-05-11T14:30:11,206 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_microeconomics.yaml' 2026-05-11T14:30:11,207 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_physics.yaml' 2026-05-11T14:30:11,208 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_psychology.yaml' 2026-05-11T14:30:11,209 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_statistics.yaml' 2026-05-11T14:30:11,211 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_us_history.yaml' 2026-05-11T14:30:11,212 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_world_history.yaml' 2026-05-11T14:30:11,213 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_aging.yaml' 2026-05-11T14:30:11,214 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_sexuality.yaml' 2026-05-11T14:30:11,216 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_international_law.yaml' 2026-05-11T14:30:11,217 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_jurisprudence.yaml' 2026-05-11T14:30:11,218 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_logical_fallacies.yaml' 2026-05-11T14:30:11,219 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_machine_learning.yaml' 2026-05-11T14:30:11,221 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_management.yaml' 2026-05-11T14:30:11,222 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_marketing.yaml' 2026-05-11T14:30:11,223 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_medical_genetics.yaml' 2026-05-11T14:30:11,224 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_miscellaneous.yaml' 2026-05-11T14:30:11,225 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_disputes.yaml' 2026-05-11T14:30:11,226 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_scenarios.yaml' 2026-05-11T14:30:11,228 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_nutrition.yaml' 2026-05-11T14:30:11,229 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_philosophy.yaml' 2026-05-11T14:30:11,230 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_prehistory.yaml' 2026-05-11T14:30:11,231 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_accounting.yaml' 2026-05-11T14:30:11,232 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_law.yaml' 2026-05-11T14:30:11,233 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_medicine.yaml' 2026-05-11T14:30:11,235 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_psychology.yaml' 2026-05-11T14:30:11,236 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_public_relations.yaml' 2026-05-11T14:30:11,237 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_security_studies.yaml' 2026-05-11T14:30:11,238 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_sociology.yaml' 2026-05-11T14:30:11,239 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_us_foreign_policy.yaml' 2026-05-11T14:30:11,240 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_virology.yaml' 2026-05-11T14:30:11,242 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_world_religions.yaml' 2026-05-11T14:30:11,243 adding 'lm_eval/tasks/global_mmlu/full/pt/utils.py' 2026-05-11T14:30:11,246 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro.yaml' 2026-05-11T14:30:11,247 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_humanities.yaml' 2026-05-11T14:30:11,248 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_other.yaml' 2026-05-11T14:30:11,249 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_social_sciences.yaml' 2026-05-11T14:30:11,250 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_stem.yaml' 2026-05-11T14:30:11,252 adding 'lm_eval/tasks/global_mmlu/full/ro/_ro_template_yaml' 2026-05-11T14:30:11,253 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_abstract_algebra.yaml' 2026-05-11T14:30:11,254 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_anatomy.yaml' 2026-05-11T14:30:11,255 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_astronomy.yaml' 2026-05-11T14:30:11,257 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_business_ethics.yaml' 2026-05-11T14:30:11,258 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_clinical_knowledge.yaml' 2026-05-11T14:30:11,259 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_biology.yaml' 2026-05-11T14:30:11,261 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_chemistry.yaml' 2026-05-11T14:30:11,262 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_computer_science.yaml' 2026-05-11T14:30:11,263 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_mathematics.yaml' 2026-05-11T14:30:11,264 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_medicine.yaml' 2026-05-11T14:30:11,266 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_physics.yaml' 2026-05-11T14:30:11,267 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_computer_security.yaml' 2026-05-11T14:30:11,268 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_conceptual_physics.yaml' 2026-05-11T14:30:11,269 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_econometrics.yaml' 2026-05-11T14:30:11,271 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_electrical_engineering.yaml' 2026-05-11T14:30:11,272 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_elementary_mathematics.yaml' 2026-05-11T14:30:11,273 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_formal_logic.yaml' 2026-05-11T14:30:11,275 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_global_facts.yaml' 2026-05-11T14:30:11,276 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_biology.yaml' 2026-05-11T14:30:11,277 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_chemistry.yaml' 2026-05-11T14:30:11,278 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_computer_science.yaml' 2026-05-11T14:30:11,279 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_european_history.yaml' 2026-05-11T14:30:11,280 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_geography.yaml' 2026-05-11T14:30:11,282 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,283 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,284 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_mathematics.yaml' 2026-05-11T14:30:11,285 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_microeconomics.yaml' 2026-05-11T14:30:11,286 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_physics.yaml' 2026-05-11T14:30:11,287 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_psychology.yaml' 2026-05-11T14:30:11,289 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_statistics.yaml' 2026-05-11T14:30:11,290 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_us_history.yaml' 2026-05-11T14:30:11,291 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_world_history.yaml' 2026-05-11T14:30:11,292 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_aging.yaml' 2026-05-11T14:30:11,293 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_sexuality.yaml' 2026-05-11T14:30:11,294 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_international_law.yaml' 2026-05-11T14:30:11,296 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_jurisprudence.yaml' 2026-05-11T14:30:11,297 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_logical_fallacies.yaml' 2026-05-11T14:30:11,298 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_machine_learning.yaml' 2026-05-11T14:30:11,299 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_management.yaml' 2026-05-11T14:30:11,300 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_marketing.yaml' 2026-05-11T14:30:11,301 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_medical_genetics.yaml' 2026-05-11T14:30:11,303 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_miscellaneous.yaml' 2026-05-11T14:30:11,304 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_disputes.yaml' 2026-05-11T14:30:11,305 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_scenarios.yaml' 2026-05-11T14:30:11,306 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_nutrition.yaml' 2026-05-11T14:30:11,307 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_philosophy.yaml' 2026-05-11T14:30:11,308 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_prehistory.yaml' 2026-05-11T14:30:11,310 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_accounting.yaml' 2026-05-11T14:30:11,311 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_law.yaml' 2026-05-11T14:30:11,312 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_medicine.yaml' 2026-05-11T14:30:11,313 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_psychology.yaml' 2026-05-11T14:30:11,314 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_public_relations.yaml' 2026-05-11T14:30:11,315 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_security_studies.yaml' 2026-05-11T14:30:11,317 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_sociology.yaml' 2026-05-11T14:30:11,318 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_us_foreign_policy.yaml' 2026-05-11T14:30:11,319 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_virology.yaml' 2026-05-11T14:30:11,320 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_world_religions.yaml' 2026-05-11T14:30:11,321 adding 'lm_eval/tasks/global_mmlu/full/ro/utils.py' 2026-05-11T14:30:11,324 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru.yaml' 2026-05-11T14:30:11,326 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_humanities.yaml' 2026-05-11T14:30:11,327 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_other.yaml' 2026-05-11T14:30:11,328 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_social_sciences.yaml' 2026-05-11T14:30:11,329 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_stem.yaml' 2026-05-11T14:30:11,331 adding 'lm_eval/tasks/global_mmlu/full/ru/_ru_template_yaml' 2026-05-11T14:30:11,332 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_abstract_algebra.yaml' 2026-05-11T14:30:11,333 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_anatomy.yaml' 2026-05-11T14:30:11,334 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_astronomy.yaml' 2026-05-11T14:30:11,335 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_business_ethics.yaml' 2026-05-11T14:30:11,337 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_clinical_knowledge.yaml' 2026-05-11T14:30:11,338 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_biology.yaml' 2026-05-11T14:30:11,339 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_chemistry.yaml' 2026-05-11T14:30:11,340 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_computer_science.yaml' 2026-05-11T14:30:11,341 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_mathematics.yaml' 2026-05-11T14:30:11,342 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_medicine.yaml' 2026-05-11T14:30:11,344 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_physics.yaml' 2026-05-11T14:30:11,345 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_computer_security.yaml' 2026-05-11T14:30:11,346 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_conceptual_physics.yaml' 2026-05-11T14:30:11,347 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_econometrics.yaml' 2026-05-11T14:30:11,348 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_electrical_engineering.yaml' 2026-05-11T14:30:11,349 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_elementary_mathematics.yaml' 2026-05-11T14:30:11,350 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_formal_logic.yaml' 2026-05-11T14:30:11,351 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_global_facts.yaml' 2026-05-11T14:30:11,352 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_biology.yaml' 2026-05-11T14:30:11,353 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_chemistry.yaml' 2026-05-11T14:30:11,355 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_computer_science.yaml' 2026-05-11T14:30:11,356 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_european_history.yaml' 2026-05-11T14:30:11,357 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_geography.yaml' 2026-05-11T14:30:11,358 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,359 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,360 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_mathematics.yaml' 2026-05-11T14:30:11,361 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_microeconomics.yaml' 2026-05-11T14:30:11,363 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_physics.yaml' 2026-05-11T14:30:11,364 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_psychology.yaml' 2026-05-11T14:30:11,365 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_statistics.yaml' 2026-05-11T14:30:11,366 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_us_history.yaml' 2026-05-11T14:30:11,367 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_world_history.yaml' 2026-05-11T14:30:11,368 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_aging.yaml' 2026-05-11T14:30:11,369 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_sexuality.yaml' 2026-05-11T14:30:11,371 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_international_law.yaml' 2026-05-11T14:30:11,372 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_jurisprudence.yaml' 2026-05-11T14:30:11,373 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_logical_fallacies.yaml' 2026-05-11T14:30:11,374 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_machine_learning.yaml' 2026-05-11T14:30:11,375 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_management.yaml' 2026-05-11T14:30:11,376 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_marketing.yaml' 2026-05-11T14:30:11,377 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_medical_genetics.yaml' 2026-05-11T14:30:11,379 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_miscellaneous.yaml' 2026-05-11T14:30:11,380 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_disputes.yaml' 2026-05-11T14:30:11,381 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_scenarios.yaml' 2026-05-11T14:30:11,382 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_nutrition.yaml' 2026-05-11T14:30:11,383 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_philosophy.yaml' 2026-05-11T14:30:11,384 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_prehistory.yaml' 2026-05-11T14:30:11,385 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_accounting.yaml' 2026-05-11T14:30:11,387 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_law.yaml' 2026-05-11T14:30:11,388 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_medicine.yaml' 2026-05-11T14:30:11,389 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_psychology.yaml' 2026-05-11T14:30:11,390 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_public_relations.yaml' 2026-05-11T14:30:11,391 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_security_studies.yaml' 2026-05-11T14:30:11,392 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_sociology.yaml' 2026-05-11T14:30:11,393 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_us_foreign_policy.yaml' 2026-05-11T14:30:11,394 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_virology.yaml' 2026-05-11T14:30:11,396 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_world_religions.yaml' 2026-05-11T14:30:11,397 adding 'lm_eval/tasks/global_mmlu/full/ru/utils.py' 2026-05-11T14:30:11,400 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si.yaml' 2026-05-11T14:30:11,401 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_humanities.yaml' 2026-05-11T14:30:11,402 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_other.yaml' 2026-05-11T14:30:11,403 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_social_sciences.yaml' 2026-05-11T14:30:11,404 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_stem.yaml' 2026-05-11T14:30:11,406 adding 'lm_eval/tasks/global_mmlu/full/si/_si_template_yaml' 2026-05-11T14:30:11,407 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_abstract_algebra.yaml' 2026-05-11T14:30:11,408 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_anatomy.yaml' 2026-05-11T14:30:11,409 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_astronomy.yaml' 2026-05-11T14:30:11,410 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_business_ethics.yaml' 2026-05-11T14:30:11,411 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_clinical_knowledge.yaml' 2026-05-11T14:30:11,412 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_biology.yaml' 2026-05-11T14:30:11,414 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_chemistry.yaml' 2026-05-11T14:30:11,415 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_computer_science.yaml' 2026-05-11T14:30:11,416 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_mathematics.yaml' 2026-05-11T14:30:11,417 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_medicine.yaml' 2026-05-11T14:30:11,418 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_physics.yaml' 2026-05-11T14:30:11,419 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_computer_security.yaml' 2026-05-11T14:30:11,421 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_conceptual_physics.yaml' 2026-05-11T14:30:11,422 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_econometrics.yaml' 2026-05-11T14:30:11,423 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_electrical_engineering.yaml' 2026-05-11T14:30:11,424 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_elementary_mathematics.yaml' 2026-05-11T14:30:11,425 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_formal_logic.yaml' 2026-05-11T14:30:11,427 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_global_facts.yaml' 2026-05-11T14:30:11,428 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_biology.yaml' 2026-05-11T14:30:11,429 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_chemistry.yaml' 2026-05-11T14:30:11,430 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_computer_science.yaml' 2026-05-11T14:30:11,432 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_european_history.yaml' 2026-05-11T14:30:11,433 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_geography.yaml' 2026-05-11T14:30:11,434 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,435 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,436 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_mathematics.yaml' 2026-05-11T14:30:11,437 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_microeconomics.yaml' 2026-05-11T14:30:11,438 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_physics.yaml' 2026-05-11T14:30:11,439 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_psychology.yaml' 2026-05-11T14:30:11,440 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_statistics.yaml' 2026-05-11T14:30:11,441 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_us_history.yaml' 2026-05-11T14:30:11,442 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_world_history.yaml' 2026-05-11T14:30:11,444 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_aging.yaml' 2026-05-11T14:30:11,445 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_sexuality.yaml' 2026-05-11T14:30:11,446 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_international_law.yaml' 2026-05-11T14:30:11,447 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_jurisprudence.yaml' 2026-05-11T14:30:11,448 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_logical_fallacies.yaml' 2026-05-11T14:30:11,449 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_machine_learning.yaml' 2026-05-11T14:30:11,450 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_management.yaml' 2026-05-11T14:30:11,451 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_marketing.yaml' 2026-05-11T14:30:11,453 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_medical_genetics.yaml' 2026-05-11T14:30:11,454 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_miscellaneous.yaml' 2026-05-11T14:30:11,455 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_disputes.yaml' 2026-05-11T14:30:11,456 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_scenarios.yaml' 2026-05-11T14:30:11,457 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_nutrition.yaml' 2026-05-11T14:30:11,458 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_philosophy.yaml' 2026-05-11T14:30:11,460 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_prehistory.yaml' 2026-05-11T14:30:11,461 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_accounting.yaml' 2026-05-11T14:30:11,462 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_law.yaml' 2026-05-11T14:30:11,463 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_medicine.yaml' 2026-05-11T14:30:11,464 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_psychology.yaml' 2026-05-11T14:30:11,466 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_public_relations.yaml' 2026-05-11T14:30:11,467 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_security_studies.yaml' 2026-05-11T14:30:11,468 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_sociology.yaml' 2026-05-11T14:30:11,469 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_us_foreign_policy.yaml' 2026-05-11T14:30:11,470 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_virology.yaml' 2026-05-11T14:30:11,472 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_world_religions.yaml' 2026-05-11T14:30:11,473 adding 'lm_eval/tasks/global_mmlu/full/si/utils.py' 2026-05-11T14:30:11,476 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn.yaml' 2026-05-11T14:30:11,478 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_humanities.yaml' 2026-05-11T14:30:11,479 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_other.yaml' 2026-05-11T14:30:11,480 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_social_sciences.yaml' 2026-05-11T14:30:11,481 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_stem.yaml' 2026-05-11T14:30:11,482 adding 'lm_eval/tasks/global_mmlu/full/sn/_sn_template_yaml' 2026-05-11T14:30:11,483 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_abstract_algebra.yaml' 2026-05-11T14:30:11,484 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_anatomy.yaml' 2026-05-11T14:30:11,485 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_astronomy.yaml' 2026-05-11T14:30:11,486 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_business_ethics.yaml' 2026-05-11T14:30:11,487 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_clinical_knowledge.yaml' 2026-05-11T14:30:11,488 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_biology.yaml' 2026-05-11T14:30:11,490 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_chemistry.yaml' 2026-05-11T14:30:11,491 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_computer_science.yaml' 2026-05-11T14:30:11,492 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_mathematics.yaml' 2026-05-11T14:30:11,493 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_medicine.yaml' 2026-05-11T14:30:11,494 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_physics.yaml' 2026-05-11T14:30:11,495 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_computer_security.yaml' 2026-05-11T14:30:11,496 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_conceptual_physics.yaml' 2026-05-11T14:30:11,497 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_econometrics.yaml' 2026-05-11T14:30:11,498 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_electrical_engineering.yaml' 2026-05-11T14:30:11,500 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_elementary_mathematics.yaml' 2026-05-11T14:30:11,501 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_formal_logic.yaml' 2026-05-11T14:30:11,502 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_global_facts.yaml' 2026-05-11T14:30:11,503 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_biology.yaml' 2026-05-11T14:30:11,504 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_chemistry.yaml' 2026-05-11T14:30:11,505 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_computer_science.yaml' 2026-05-11T14:30:11,507 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_european_history.yaml' 2026-05-11T14:30:11,508 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_geography.yaml' 2026-05-11T14:30:11,509 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,510 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,511 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_mathematics.yaml' 2026-05-11T14:30:11,512 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_microeconomics.yaml' 2026-05-11T14:30:11,514 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_physics.yaml' 2026-05-11T14:30:11,515 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_psychology.yaml' 2026-05-11T14:30:11,516 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_statistics.yaml' 2026-05-11T14:30:11,517 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_us_history.yaml' 2026-05-11T14:30:11,518 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_world_history.yaml' 2026-05-11T14:30:11,519 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_aging.yaml' 2026-05-11T14:30:11,521 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_sexuality.yaml' 2026-05-11T14:30:11,522 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_international_law.yaml' 2026-05-11T14:30:11,523 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_jurisprudence.yaml' 2026-05-11T14:30:11,524 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_logical_fallacies.yaml' 2026-05-11T14:30:11,525 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_machine_learning.yaml' 2026-05-11T14:30:11,526 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_management.yaml' 2026-05-11T14:30:11,527 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_marketing.yaml' 2026-05-11T14:30:11,528 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_medical_genetics.yaml' 2026-05-11T14:30:11,529 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_miscellaneous.yaml' 2026-05-11T14:30:11,530 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_disputes.yaml' 2026-05-11T14:30:11,531 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_scenarios.yaml' 2026-05-11T14:30:11,533 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_nutrition.yaml' 2026-05-11T14:30:11,534 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_philosophy.yaml' 2026-05-11T14:30:11,535 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_prehistory.yaml' 2026-05-11T14:30:11,536 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_accounting.yaml' 2026-05-11T14:30:11,537 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_law.yaml' 2026-05-11T14:30:11,538 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_medicine.yaml' 2026-05-11T14:30:11,539 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_psychology.yaml' 2026-05-11T14:30:11,540 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_public_relations.yaml' 2026-05-11T14:30:11,542 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_security_studies.yaml' 2026-05-11T14:30:11,543 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_sociology.yaml' 2026-05-11T14:30:11,544 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_us_foreign_policy.yaml' 2026-05-11T14:30:11,545 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_virology.yaml' 2026-05-11T14:30:11,546 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_world_religions.yaml' 2026-05-11T14:30:11,547 adding 'lm_eval/tasks/global_mmlu/full/sn/utils.py' 2026-05-11T14:30:11,550 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so.yaml' 2026-05-11T14:30:11,551 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_humanities.yaml' 2026-05-11T14:30:11,553 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_other.yaml' 2026-05-11T14:30:11,554 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_social_sciences.yaml' 2026-05-11T14:30:11,555 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_stem.yaml' 2026-05-11T14:30:11,556 adding 'lm_eval/tasks/global_mmlu/full/so/_so_template_yaml' 2026-05-11T14:30:11,557 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_abstract_algebra.yaml' 2026-05-11T14:30:11,558 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_anatomy.yaml' 2026-05-11T14:30:11,560 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_astronomy.yaml' 2026-05-11T14:30:11,561 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_business_ethics.yaml' 2026-05-11T14:30:11,562 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_clinical_knowledge.yaml' 2026-05-11T14:30:11,563 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_biology.yaml' 2026-05-11T14:30:11,564 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_chemistry.yaml' 2026-05-11T14:30:11,565 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_computer_science.yaml' 2026-05-11T14:30:11,566 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_mathematics.yaml' 2026-05-11T14:30:11,568 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_medicine.yaml' 2026-05-11T14:30:11,569 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_physics.yaml' 2026-05-11T14:30:11,570 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_computer_security.yaml' 2026-05-11T14:30:11,571 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_conceptual_physics.yaml' 2026-05-11T14:30:11,572 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_econometrics.yaml' 2026-05-11T14:30:11,573 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_electrical_engineering.yaml' 2026-05-11T14:30:11,574 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_elementary_mathematics.yaml' 2026-05-11T14:30:11,575 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_formal_logic.yaml' 2026-05-11T14:30:11,576 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_global_facts.yaml' 2026-05-11T14:30:11,577 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_biology.yaml' 2026-05-11T14:30:11,578 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_chemistry.yaml' 2026-05-11T14:30:11,579 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_computer_science.yaml' 2026-05-11T14:30:11,581 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_european_history.yaml' 2026-05-11T14:30:11,582 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_geography.yaml' 2026-05-11T14:30:11,583 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,584 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,585 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_mathematics.yaml' 2026-05-11T14:30:11,586 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_microeconomics.yaml' 2026-05-11T14:30:11,587 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_physics.yaml' 2026-05-11T14:30:11,588 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_psychology.yaml' 2026-05-11T14:30:11,590 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_statistics.yaml' 2026-05-11T14:30:11,591 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_us_history.yaml' 2026-05-11T14:30:11,592 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_world_history.yaml' 2026-05-11T14:30:11,593 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_aging.yaml' 2026-05-11T14:30:11,594 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_sexuality.yaml' 2026-05-11T14:30:11,595 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_international_law.yaml' 2026-05-11T14:30:11,597 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_jurisprudence.yaml' 2026-05-11T14:30:11,598 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_logical_fallacies.yaml' 2026-05-11T14:30:11,599 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_machine_learning.yaml' 2026-05-11T14:30:11,600 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_management.yaml' 2026-05-11T14:30:11,601 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_marketing.yaml' 2026-05-11T14:30:11,603 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_medical_genetics.yaml' 2026-05-11T14:30:11,604 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_miscellaneous.yaml' 2026-05-11T14:30:11,605 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_disputes.yaml' 2026-05-11T14:30:11,606 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_scenarios.yaml' 2026-05-11T14:30:11,607 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_nutrition.yaml' 2026-05-11T14:30:11,609 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_philosophy.yaml' 2026-05-11T14:30:11,610 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_prehistory.yaml' 2026-05-11T14:30:11,611 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_accounting.yaml' 2026-05-11T14:30:11,612 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_law.yaml' 2026-05-11T14:30:11,613 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_medicine.yaml' 2026-05-11T14:30:11,614 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_psychology.yaml' 2026-05-11T14:30:11,615 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_public_relations.yaml' 2026-05-11T14:30:11,616 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_security_studies.yaml' 2026-05-11T14:30:11,617 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_sociology.yaml' 2026-05-11T14:30:11,618 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_us_foreign_policy.yaml' 2026-05-11T14:30:11,619 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_virology.yaml' 2026-05-11T14:30:11,620 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_world_religions.yaml' 2026-05-11T14:30:11,621 adding 'lm_eval/tasks/global_mmlu/full/so/utils.py' 2026-05-11T14:30:11,624 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr.yaml' 2026-05-11T14:30:11,625 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_humanities.yaml' 2026-05-11T14:30:11,626 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_other.yaml' 2026-05-11T14:30:11,627 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_social_sciences.yaml' 2026-05-11T14:30:11,629 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_stem.yaml' 2026-05-11T14:30:11,630 adding 'lm_eval/tasks/global_mmlu/full/sr/_sr_template_yaml' 2026-05-11T14:30:11,631 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_abstract_algebra.yaml' 2026-05-11T14:30:11,632 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_anatomy.yaml' 2026-05-11T14:30:11,633 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_astronomy.yaml' 2026-05-11T14:30:11,634 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_business_ethics.yaml' 2026-05-11T14:30:11,636 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_clinical_knowledge.yaml' 2026-05-11T14:30:11,637 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_biology.yaml' 2026-05-11T14:30:11,638 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_chemistry.yaml' 2026-05-11T14:30:11,639 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_computer_science.yaml' 2026-05-11T14:30:11,640 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_mathematics.yaml' 2026-05-11T14:30:11,641 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_medicine.yaml' 2026-05-11T14:30:11,643 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_physics.yaml' 2026-05-11T14:30:11,644 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_computer_security.yaml' 2026-05-11T14:30:11,645 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_conceptual_physics.yaml' 2026-05-11T14:30:11,646 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_econometrics.yaml' 2026-05-11T14:30:11,647 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_electrical_engineering.yaml' 2026-05-11T14:30:11,648 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_elementary_mathematics.yaml' 2026-05-11T14:30:11,650 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_formal_logic.yaml' 2026-05-11T14:30:11,651 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_global_facts.yaml' 2026-05-11T14:30:11,652 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_biology.yaml' 2026-05-11T14:30:11,653 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_chemistry.yaml' 2026-05-11T14:30:11,654 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_computer_science.yaml' 2026-05-11T14:30:11,655 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_european_history.yaml' 2026-05-11T14:30:11,656 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_geography.yaml' 2026-05-11T14:30:11,657 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,658 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,659 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_mathematics.yaml' 2026-05-11T14:30:11,660 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_microeconomics.yaml' 2026-05-11T14:30:11,662 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_physics.yaml' 2026-05-11T14:30:11,663 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_psychology.yaml' 2026-05-11T14:30:11,664 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_statistics.yaml' 2026-05-11T14:30:11,665 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_us_history.yaml' 2026-05-11T14:30:11,666 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_world_history.yaml' 2026-05-11T14:30:11,667 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_aging.yaml' 2026-05-11T14:30:11,668 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_sexuality.yaml' 2026-05-11T14:30:11,669 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_international_law.yaml' 2026-05-11T14:30:11,671 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_jurisprudence.yaml' 2026-05-11T14:30:11,672 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_logical_fallacies.yaml' 2026-05-11T14:30:11,673 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_machine_learning.yaml' 2026-05-11T14:30:11,674 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_management.yaml' 2026-05-11T14:30:11,675 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_marketing.yaml' 2026-05-11T14:30:11,676 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_medical_genetics.yaml' 2026-05-11T14:30:11,677 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_miscellaneous.yaml' 2026-05-11T14:30:11,678 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_disputes.yaml' 2026-05-11T14:30:11,679 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_scenarios.yaml' 2026-05-11T14:30:11,681 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_nutrition.yaml' 2026-05-11T14:30:11,682 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_philosophy.yaml' 2026-05-11T14:30:11,683 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_prehistory.yaml' 2026-05-11T14:30:11,684 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_accounting.yaml' 2026-05-11T14:30:11,685 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_law.yaml' 2026-05-11T14:30:11,686 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_medicine.yaml' 2026-05-11T14:30:11,688 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_psychology.yaml' 2026-05-11T14:30:11,689 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_public_relations.yaml' 2026-05-11T14:30:11,690 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_security_studies.yaml' 2026-05-11T14:30:11,691 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_sociology.yaml' 2026-05-11T14:30:11,692 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_us_foreign_policy.yaml' 2026-05-11T14:30:11,694 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_virology.yaml' 2026-05-11T14:30:11,695 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_world_religions.yaml' 2026-05-11T14:30:11,696 adding 'lm_eval/tasks/global_mmlu/full/sr/utils.py' 2026-05-11T14:30:11,698 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv.yaml' 2026-05-11T14:30:11,699 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_humanities.yaml' 2026-05-11T14:30:11,701 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_other.yaml' 2026-05-11T14:30:11,702 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_social_sciences.yaml' 2026-05-11T14:30:11,703 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_stem.yaml' 2026-05-11T14:30:11,704 adding 'lm_eval/tasks/global_mmlu/full/sv/_sv_template_yaml' 2026-05-11T14:30:11,705 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_abstract_algebra.yaml' 2026-05-11T14:30:11,706 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_anatomy.yaml' 2026-05-11T14:30:11,707 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_astronomy.yaml' 2026-05-11T14:30:11,708 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_business_ethics.yaml' 2026-05-11T14:30:11,710 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_clinical_knowledge.yaml' 2026-05-11T14:30:11,711 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_biology.yaml' 2026-05-11T14:30:11,712 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_chemistry.yaml' 2026-05-11T14:30:11,713 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_computer_science.yaml' 2026-05-11T14:30:11,714 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_mathematics.yaml' 2026-05-11T14:30:11,715 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_medicine.yaml' 2026-05-11T14:30:11,716 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_physics.yaml' 2026-05-11T14:30:11,717 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_computer_security.yaml' 2026-05-11T14:30:11,718 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_conceptual_physics.yaml' 2026-05-11T14:30:11,720 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_econometrics.yaml' 2026-05-11T14:30:11,721 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_electrical_engineering.yaml' 2026-05-11T14:30:11,722 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_elementary_mathematics.yaml' 2026-05-11T14:30:11,723 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_formal_logic.yaml' 2026-05-11T14:30:11,724 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_global_facts.yaml' 2026-05-11T14:30:11,725 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_biology.yaml' 2026-05-11T14:30:11,726 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_chemistry.yaml' 2026-05-11T14:30:11,728 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_computer_science.yaml' 2026-05-11T14:30:11,729 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_european_history.yaml' 2026-05-11T14:30:11,730 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_geography.yaml' 2026-05-11T14:30:11,731 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,732 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,733 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_mathematics.yaml' 2026-05-11T14:30:11,735 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_microeconomics.yaml' 2026-05-11T14:30:11,736 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_physics.yaml' 2026-05-11T14:30:11,737 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_psychology.yaml' 2026-05-11T14:30:11,738 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_statistics.yaml' 2026-05-11T14:30:11,739 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_us_history.yaml' 2026-05-11T14:30:11,740 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_world_history.yaml' 2026-05-11T14:30:11,741 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_aging.yaml' 2026-05-11T14:30:11,742 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_sexuality.yaml' 2026-05-11T14:30:11,743 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_international_law.yaml' 2026-05-11T14:30:11,744 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_jurisprudence.yaml' 2026-05-11T14:30:11,745 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_logical_fallacies.yaml' 2026-05-11T14:30:11,746 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_machine_learning.yaml' 2026-05-11T14:30:11,748 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_management.yaml' 2026-05-11T14:30:11,749 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_marketing.yaml' 2026-05-11T14:30:11,750 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_medical_genetics.yaml' 2026-05-11T14:30:11,751 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_miscellaneous.yaml' 2026-05-11T14:30:11,752 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_disputes.yaml' 2026-05-11T14:30:11,753 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_scenarios.yaml' 2026-05-11T14:30:11,754 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_nutrition.yaml' 2026-05-11T14:30:11,756 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_philosophy.yaml' 2026-05-11T14:30:11,757 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_prehistory.yaml' 2026-05-11T14:30:11,758 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_accounting.yaml' 2026-05-11T14:30:11,759 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_law.yaml' 2026-05-11T14:30:11,760 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_medicine.yaml' 2026-05-11T14:30:11,761 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_psychology.yaml' 2026-05-11T14:30:11,762 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_public_relations.yaml' 2026-05-11T14:30:11,764 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_security_studies.yaml' 2026-05-11T14:30:11,765 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_sociology.yaml' 2026-05-11T14:30:11,766 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_us_foreign_policy.yaml' 2026-05-11T14:30:11,767 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_virology.yaml' 2026-05-11T14:30:11,768 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_world_religions.yaml' 2026-05-11T14:30:11,769 adding 'lm_eval/tasks/global_mmlu/full/sv/utils.py' 2026-05-11T14:30:11,772 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw.yaml' 2026-05-11T14:30:11,774 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_humanities.yaml' 2026-05-11T14:30:11,775 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_other.yaml' 2026-05-11T14:30:11,776 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_social_sciences.yaml' 2026-05-11T14:30:11,777 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_stem.yaml' 2026-05-11T14:30:11,778 adding 'lm_eval/tasks/global_mmlu/full/sw/_sw_template_yaml' 2026-05-11T14:30:11,779 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_abstract_algebra.yaml' 2026-05-11T14:30:11,780 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_anatomy.yaml' 2026-05-11T14:30:11,782 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_astronomy.yaml' 2026-05-11T14:30:11,783 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_business_ethics.yaml' 2026-05-11T14:30:11,784 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_clinical_knowledge.yaml' 2026-05-11T14:30:11,785 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_biology.yaml' 2026-05-11T14:30:11,786 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_chemistry.yaml' 2026-05-11T14:30:11,787 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_computer_science.yaml' 2026-05-11T14:30:11,788 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_mathematics.yaml' 2026-05-11T14:30:11,789 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_medicine.yaml' 2026-05-11T14:30:11,790 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_physics.yaml' 2026-05-11T14:30:11,791 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_computer_security.yaml' 2026-05-11T14:30:11,792 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_conceptual_physics.yaml' 2026-05-11T14:30:11,794 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_econometrics.yaml' 2026-05-11T14:30:11,795 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_electrical_engineering.yaml' 2026-05-11T14:30:11,796 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_elementary_mathematics.yaml' 2026-05-11T14:30:11,797 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_formal_logic.yaml' 2026-05-11T14:30:11,799 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_global_facts.yaml' 2026-05-11T14:30:11,800 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_biology.yaml' 2026-05-11T14:30:11,801 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_chemistry.yaml' 2026-05-11T14:30:11,803 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_computer_science.yaml' 2026-05-11T14:30:11,804 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_european_history.yaml' 2026-05-11T14:30:11,805 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_geography.yaml' 2026-05-11T14:30:11,806 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,807 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,808 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_mathematics.yaml' 2026-05-11T14:30:11,809 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_microeconomics.yaml' 2026-05-11T14:30:11,811 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_physics.yaml' 2026-05-11T14:30:11,812 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_psychology.yaml' 2026-05-11T14:30:11,813 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_statistics.yaml' 2026-05-11T14:30:11,814 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_us_history.yaml' 2026-05-11T14:30:11,815 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_world_history.yaml' 2026-05-11T14:30:11,817 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_aging.yaml' 2026-05-11T14:30:11,818 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_sexuality.yaml' 2026-05-11T14:30:11,819 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_international_law.yaml' 2026-05-11T14:30:11,820 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_jurisprudence.yaml' 2026-05-11T14:30:11,821 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_logical_fallacies.yaml' 2026-05-11T14:30:11,823 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_machine_learning.yaml' 2026-05-11T14:30:11,824 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_management.yaml' 2026-05-11T14:30:11,825 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_marketing.yaml' 2026-05-11T14:30:11,826 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_medical_genetics.yaml' 2026-05-11T14:30:11,827 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_miscellaneous.yaml' 2026-05-11T14:30:11,828 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_disputes.yaml' 2026-05-11T14:30:11,829 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_scenarios.yaml' 2026-05-11T14:30:11,831 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_nutrition.yaml' 2026-05-11T14:30:11,832 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_philosophy.yaml' 2026-05-11T14:30:11,833 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_prehistory.yaml' 2026-05-11T14:30:11,834 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_accounting.yaml' 2026-05-11T14:30:11,835 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_law.yaml' 2026-05-11T14:30:11,836 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_medicine.yaml' 2026-05-11T14:30:11,837 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_psychology.yaml' 2026-05-11T14:30:11,839 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_public_relations.yaml' 2026-05-11T14:30:11,840 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_security_studies.yaml' 2026-05-11T14:30:11,841 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_sociology.yaml' 2026-05-11T14:30:11,842 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_us_foreign_policy.yaml' 2026-05-11T14:30:11,843 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_virology.yaml' 2026-05-11T14:30:11,844 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_world_religions.yaml' 2026-05-11T14:30:11,846 adding 'lm_eval/tasks/global_mmlu/full/sw/utils.py' 2026-05-11T14:30:11,849 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te.yaml' 2026-05-11T14:30:11,850 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_humanities.yaml' 2026-05-11T14:30:11,851 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_other.yaml' 2026-05-11T14:30:11,852 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_social_sciences.yaml' 2026-05-11T14:30:11,853 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_stem.yaml' 2026-05-11T14:30:11,854 adding 'lm_eval/tasks/global_mmlu/full/te/_te_template_yaml' 2026-05-11T14:30:11,855 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_abstract_algebra.yaml' 2026-05-11T14:30:11,857 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_anatomy.yaml' 2026-05-11T14:30:11,858 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_astronomy.yaml' 2026-05-11T14:30:11,859 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_business_ethics.yaml' 2026-05-11T14:30:11,860 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_clinical_knowledge.yaml' 2026-05-11T14:30:11,861 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_biology.yaml' 2026-05-11T14:30:11,862 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_chemistry.yaml' 2026-05-11T14:30:11,864 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_computer_science.yaml' 2026-05-11T14:30:11,865 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_mathematics.yaml' 2026-05-11T14:30:11,866 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_medicine.yaml' 2026-05-11T14:30:11,867 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_physics.yaml' 2026-05-11T14:30:11,868 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_computer_security.yaml' 2026-05-11T14:30:11,870 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_conceptual_physics.yaml' 2026-05-11T14:30:11,871 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_econometrics.yaml' 2026-05-11T14:30:11,872 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_electrical_engineering.yaml' 2026-05-11T14:30:11,873 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_elementary_mathematics.yaml' 2026-05-11T14:30:11,874 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_formal_logic.yaml' 2026-05-11T14:30:11,875 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_global_facts.yaml' 2026-05-11T14:30:11,876 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_biology.yaml' 2026-05-11T14:30:11,877 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_chemistry.yaml' 2026-05-11T14:30:11,878 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_computer_science.yaml' 2026-05-11T14:30:11,879 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_european_history.yaml' 2026-05-11T14:30:11,880 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_geography.yaml' 2026-05-11T14:30:11,881 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,883 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,884 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_mathematics.yaml' 2026-05-11T14:30:11,885 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_microeconomics.yaml' 2026-05-11T14:30:11,886 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_physics.yaml' 2026-05-11T14:30:11,887 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_psychology.yaml' 2026-05-11T14:30:11,888 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_statistics.yaml' 2026-05-11T14:30:11,889 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_us_history.yaml' 2026-05-11T14:30:11,890 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_world_history.yaml' 2026-05-11T14:30:11,892 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_aging.yaml' 2026-05-11T14:30:11,893 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_sexuality.yaml' 2026-05-11T14:30:11,894 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_international_law.yaml' 2026-05-11T14:30:11,895 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_jurisprudence.yaml' 2026-05-11T14:30:11,896 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_logical_fallacies.yaml' 2026-05-11T14:30:11,897 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_machine_learning.yaml' 2026-05-11T14:30:11,899 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_management.yaml' 2026-05-11T14:30:11,900 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_marketing.yaml' 2026-05-11T14:30:11,901 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_medical_genetics.yaml' 2026-05-11T14:30:11,902 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_miscellaneous.yaml' 2026-05-11T14:30:11,904 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_disputes.yaml' 2026-05-11T14:30:11,905 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_scenarios.yaml' 2026-05-11T14:30:11,906 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_nutrition.yaml' 2026-05-11T14:30:11,908 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_philosophy.yaml' 2026-05-11T14:30:11,909 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_prehistory.yaml' 2026-05-11T14:30:11,910 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_accounting.yaml' 2026-05-11T14:30:11,911 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_law.yaml' 2026-05-11T14:30:11,912 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_medicine.yaml' 2026-05-11T14:30:11,913 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_psychology.yaml' 2026-05-11T14:30:11,914 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_public_relations.yaml' 2026-05-11T14:30:11,915 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_security_studies.yaml' 2026-05-11T14:30:11,916 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_sociology.yaml' 2026-05-11T14:30:11,917 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_us_foreign_policy.yaml' 2026-05-11T14:30:11,918 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_virology.yaml' 2026-05-11T14:30:11,919 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_world_religions.yaml' 2026-05-11T14:30:11,921 adding 'lm_eval/tasks/global_mmlu/full/te/utils.py' 2026-05-11T14:30:11,923 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr.yaml' 2026-05-11T14:30:11,924 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_humanities.yaml' 2026-05-11T14:30:11,926 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_other.yaml' 2026-05-11T14:30:11,927 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_social_sciences.yaml' 2026-05-11T14:30:11,928 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_stem.yaml' 2026-05-11T14:30:11,929 adding 'lm_eval/tasks/global_mmlu/full/tr/_tr_template_yaml' 2026-05-11T14:30:11,930 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_abstract_algebra.yaml' 2026-05-11T14:30:11,931 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_anatomy.yaml' 2026-05-11T14:30:11,933 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_astronomy.yaml' 2026-05-11T14:30:11,934 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_business_ethics.yaml' 2026-05-11T14:30:11,935 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_clinical_knowledge.yaml' 2026-05-11T14:30:11,936 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_biology.yaml' 2026-05-11T14:30:11,937 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_chemistry.yaml' 2026-05-11T14:30:11,938 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_computer_science.yaml' 2026-05-11T14:30:11,939 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_mathematics.yaml' 2026-05-11T14:30:11,941 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_medicine.yaml' 2026-05-11T14:30:11,942 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_physics.yaml' 2026-05-11T14:30:11,943 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_computer_security.yaml' 2026-05-11T14:30:11,944 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_conceptual_physics.yaml' 2026-05-11T14:30:11,945 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_econometrics.yaml' 2026-05-11T14:30:11,946 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_electrical_engineering.yaml' 2026-05-11T14:30:11,947 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_elementary_mathematics.yaml' 2026-05-11T14:30:11,949 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_formal_logic.yaml' 2026-05-11T14:30:11,950 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_global_facts.yaml' 2026-05-11T14:30:11,951 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_biology.yaml' 2026-05-11T14:30:11,952 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_chemistry.yaml' 2026-05-11T14:30:11,953 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_computer_science.yaml' 2026-05-11T14:30:11,954 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_european_history.yaml' 2026-05-11T14:30:11,956 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_geography.yaml' 2026-05-11T14:30:11,957 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_government_and_politics.yaml' 2026-05-11T14:30:11,958 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_macroeconomics.yaml' 2026-05-11T14:30:11,959 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_mathematics.yaml' 2026-05-11T14:30:11,960 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_microeconomics.yaml' 2026-05-11T14:30:11,961 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_physics.yaml' 2026-05-11T14:30:11,962 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_psychology.yaml' 2026-05-11T14:30:11,963 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_statistics.yaml' 2026-05-11T14:30:11,964 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_us_history.yaml' 2026-05-11T14:30:11,966 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_world_history.yaml' 2026-05-11T14:30:11,967 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_aging.yaml' 2026-05-11T14:30:11,968 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_sexuality.yaml' 2026-05-11T14:30:11,969 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_international_law.yaml' 2026-05-11T14:30:11,970 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_jurisprudence.yaml' 2026-05-11T14:30:11,971 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_logical_fallacies.yaml' 2026-05-11T14:30:11,972 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_machine_learning.yaml' 2026-05-11T14:30:11,973 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_management.yaml' 2026-05-11T14:30:11,975 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_marketing.yaml' 2026-05-11T14:30:11,976 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_medical_genetics.yaml' 2026-05-11T14:30:11,977 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_miscellaneous.yaml' 2026-05-11T14:30:11,978 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_disputes.yaml' 2026-05-11T14:30:11,979 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_scenarios.yaml' 2026-05-11T14:30:11,980 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_nutrition.yaml' 2026-05-11T14:30:11,981 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_philosophy.yaml' 2026-05-11T14:30:11,983 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_prehistory.yaml' 2026-05-11T14:30:11,984 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_accounting.yaml' 2026-05-11T14:30:11,985 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_law.yaml' 2026-05-11T14:30:11,986 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_medicine.yaml' 2026-05-11T14:30:11,987 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_psychology.yaml' 2026-05-11T14:30:11,988 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_public_relations.yaml' 2026-05-11T14:30:11,989 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_security_studies.yaml' 2026-05-11T14:30:11,991 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_sociology.yaml' 2026-05-11T14:30:11,992 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_us_foreign_policy.yaml' 2026-05-11T14:30:11,993 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_virology.yaml' 2026-05-11T14:30:11,994 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_world_religions.yaml' 2026-05-11T14:30:11,995 adding 'lm_eval/tasks/global_mmlu/full/tr/utils.py' 2026-05-11T14:30:11,998 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk.yaml' 2026-05-11T14:30:11,999 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_humanities.yaml' 2026-05-11T14:30:12,000 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_other.yaml' 2026-05-11T14:30:12,002 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_social_sciences.yaml' 2026-05-11T14:30:12,003 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_stem.yaml' 2026-05-11T14:30:12,004 adding 'lm_eval/tasks/global_mmlu/full/uk/_uk_template_yaml' 2026-05-11T14:30:12,005 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_abstract_algebra.yaml' 2026-05-11T14:30:12,006 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_anatomy.yaml' 2026-05-11T14:30:12,007 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_astronomy.yaml' 2026-05-11T14:30:12,008 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_business_ethics.yaml' 2026-05-11T14:30:12,009 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_clinical_knowledge.yaml' 2026-05-11T14:30:12,011 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_biology.yaml' 2026-05-11T14:30:12,012 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_chemistry.yaml' 2026-05-11T14:30:12,013 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_computer_science.yaml' 2026-05-11T14:30:12,014 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_mathematics.yaml' 2026-05-11T14:30:12,015 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_medicine.yaml' 2026-05-11T14:30:12,016 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_physics.yaml' 2026-05-11T14:30:12,017 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_computer_security.yaml' 2026-05-11T14:30:12,018 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_conceptual_physics.yaml' 2026-05-11T14:30:12,019 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_econometrics.yaml' 2026-05-11T14:30:12,021 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_electrical_engineering.yaml' 2026-05-11T14:30:12,022 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_elementary_mathematics.yaml' 2026-05-11T14:30:12,023 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_formal_logic.yaml' 2026-05-11T14:30:12,024 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_global_facts.yaml' 2026-05-11T14:30:12,025 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_biology.yaml' 2026-05-11T14:30:12,026 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_chemistry.yaml' 2026-05-11T14:30:12,028 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_computer_science.yaml' 2026-05-11T14:30:12,029 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_european_history.yaml' 2026-05-11T14:30:12,030 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_geography.yaml' 2026-05-11T14:30:12,031 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_government_and_politics.yaml' 2026-05-11T14:30:12,032 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_macroeconomics.yaml' 2026-05-11T14:30:12,033 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_mathematics.yaml' 2026-05-11T14:30:12,035 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_microeconomics.yaml' 2026-05-11T14:30:12,036 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_physics.yaml' 2026-05-11T14:30:12,037 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_psychology.yaml' 2026-05-11T14:30:12,038 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_statistics.yaml' 2026-05-11T14:30:12,039 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_us_history.yaml' 2026-05-11T14:30:12,040 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_world_history.yaml' 2026-05-11T14:30:12,041 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_aging.yaml' 2026-05-11T14:30:12,042 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_sexuality.yaml' 2026-05-11T14:30:12,044 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_international_law.yaml' 2026-05-11T14:30:12,045 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_jurisprudence.yaml' 2026-05-11T14:30:12,046 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_logical_fallacies.yaml' 2026-05-11T14:30:12,047 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_machine_learning.yaml' 2026-05-11T14:30:12,048 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_management.yaml' 2026-05-11T14:30:12,049 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_marketing.yaml' 2026-05-11T14:30:12,050 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_medical_genetics.yaml' 2026-05-11T14:30:12,051 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_miscellaneous.yaml' 2026-05-11T14:30:12,052 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_disputes.yaml' 2026-05-11T14:30:12,053 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_scenarios.yaml' 2026-05-11T14:30:12,054 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_nutrition.yaml' 2026-05-11T14:30:12,056 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_philosophy.yaml' 2026-05-11T14:30:12,057 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_prehistory.yaml' 2026-05-11T14:30:12,058 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_accounting.yaml' 2026-05-11T14:30:12,059 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_law.yaml' 2026-05-11T14:30:12,060 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_medicine.yaml' 2026-05-11T14:30:12,061 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_psychology.yaml' 2026-05-11T14:30:12,062 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_public_relations.yaml' 2026-05-11T14:30:12,064 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_security_studies.yaml' 2026-05-11T14:30:12,065 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_sociology.yaml' 2026-05-11T14:30:12,066 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_us_foreign_policy.yaml' 2026-05-11T14:30:12,067 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_virology.yaml' 2026-05-11T14:30:12,069 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_world_religions.yaml' 2026-05-11T14:30:12,070 adding 'lm_eval/tasks/global_mmlu/full/uk/utils.py' 2026-05-11T14:30:12,073 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi.yaml' 2026-05-11T14:30:12,074 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_humanities.yaml' 2026-05-11T14:30:12,075 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_other.yaml' 2026-05-11T14:30:12,077 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_social_sciences.yaml' 2026-05-11T14:30:12,078 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_stem.yaml' 2026-05-11T14:30:12,079 adding 'lm_eval/tasks/global_mmlu/full/vi/_vi_template_yaml' 2026-05-11T14:30:12,080 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_abstract_algebra.yaml' 2026-05-11T14:30:12,081 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_anatomy.yaml' 2026-05-11T14:30:12,082 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_astronomy.yaml' 2026-05-11T14:30:12,084 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_business_ethics.yaml' 2026-05-11T14:30:12,085 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_clinical_knowledge.yaml' 2026-05-11T14:30:12,086 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_biology.yaml' 2026-05-11T14:30:12,087 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_chemistry.yaml' 2026-05-11T14:30:12,088 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_computer_science.yaml' 2026-05-11T14:30:12,089 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_mathematics.yaml' 2026-05-11T14:30:12,090 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_medicine.yaml' 2026-05-11T14:30:12,091 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_physics.yaml' 2026-05-11T14:30:12,093 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_computer_security.yaml' 2026-05-11T14:30:12,094 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_conceptual_physics.yaml' 2026-05-11T14:30:12,095 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_econometrics.yaml' 2026-05-11T14:30:12,096 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_electrical_engineering.yaml' 2026-05-11T14:30:12,097 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_elementary_mathematics.yaml' 2026-05-11T14:30:12,098 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_formal_logic.yaml' 2026-05-11T14:30:12,099 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_global_facts.yaml' 2026-05-11T14:30:12,100 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_biology.yaml' 2026-05-11T14:30:12,101 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_chemistry.yaml' 2026-05-11T14:30:12,103 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_computer_science.yaml' 2026-05-11T14:30:12,104 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_european_history.yaml' 2026-05-11T14:30:12,105 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_geography.yaml' 2026-05-11T14:30:12,106 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_government_and_politics.yaml' 2026-05-11T14:30:12,107 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_macroeconomics.yaml' 2026-05-11T14:30:12,109 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_mathematics.yaml' 2026-05-11T14:30:12,110 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_microeconomics.yaml' 2026-05-11T14:30:12,111 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_physics.yaml' 2026-05-11T14:30:12,112 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_psychology.yaml' 2026-05-11T14:30:12,113 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_statistics.yaml' 2026-05-11T14:30:12,114 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_us_history.yaml' 2026-05-11T14:30:12,116 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_world_history.yaml' 2026-05-11T14:30:12,117 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_aging.yaml' 2026-05-11T14:30:12,118 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_sexuality.yaml' 2026-05-11T14:30:12,119 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_international_law.yaml' 2026-05-11T14:30:12,120 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_jurisprudence.yaml' 2026-05-11T14:30:12,122 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_logical_fallacies.yaml' 2026-05-11T14:30:12,123 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_machine_learning.yaml' 2026-05-11T14:30:12,124 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_management.yaml' 2026-05-11T14:30:12,125 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_marketing.yaml' 2026-05-11T14:30:12,126 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_medical_genetics.yaml' 2026-05-11T14:30:12,128 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_miscellaneous.yaml' 2026-05-11T14:30:12,129 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_disputes.yaml' 2026-05-11T14:30:12,130 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_scenarios.yaml' 2026-05-11T14:30:12,131 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_nutrition.yaml' 2026-05-11T14:30:12,132 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_philosophy.yaml' 2026-05-11T14:30:12,133 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_prehistory.yaml' 2026-05-11T14:30:12,134 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_accounting.yaml' 2026-05-11T14:30:12,135 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_law.yaml' 2026-05-11T14:30:12,136 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_medicine.yaml' 2026-05-11T14:30:12,137 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_psychology.yaml' 2026-05-11T14:30:12,138 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_public_relations.yaml' 2026-05-11T14:30:12,140 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_security_studies.yaml' 2026-05-11T14:30:12,141 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_sociology.yaml' 2026-05-11T14:30:12,142 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_us_foreign_policy.yaml' 2026-05-11T14:30:12,143 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_virology.yaml' 2026-05-11T14:30:12,144 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_world_religions.yaml' 2026-05-11T14:30:12,145 adding 'lm_eval/tasks/global_mmlu/full/vi/utils.py' 2026-05-11T14:30:12,148 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo.yaml' 2026-05-11T14:30:12,150 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_humanities.yaml' 2026-05-11T14:30:12,151 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_other.yaml' 2026-05-11T14:30:12,152 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_social_sciences.yaml' 2026-05-11T14:30:12,153 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_stem.yaml' 2026-05-11T14:30:12,154 adding 'lm_eval/tasks/global_mmlu/full/yo/_yo_template_yaml' 2026-05-11T14:30:12,155 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_abstract_algebra.yaml' 2026-05-11T14:30:12,157 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_anatomy.yaml' 2026-05-11T14:30:12,158 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_astronomy.yaml' 2026-05-11T14:30:12,159 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_business_ethics.yaml' 2026-05-11T14:30:12,160 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_clinical_knowledge.yaml' 2026-05-11T14:30:12,161 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_biology.yaml' 2026-05-11T14:30:12,162 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_chemistry.yaml' 2026-05-11T14:30:12,164 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_computer_science.yaml' 2026-05-11T14:30:12,165 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_mathematics.yaml' 2026-05-11T14:30:12,166 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_medicine.yaml' 2026-05-11T14:30:12,167 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_physics.yaml' 2026-05-11T14:30:12,168 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_computer_security.yaml' 2026-05-11T14:30:12,169 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_conceptual_physics.yaml' 2026-05-11T14:30:12,171 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_econometrics.yaml' 2026-05-11T14:30:12,172 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_electrical_engineering.yaml' 2026-05-11T14:30:12,173 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_elementary_mathematics.yaml' 2026-05-11T14:30:12,174 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_formal_logic.yaml' 2026-05-11T14:30:12,175 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_global_facts.yaml' 2026-05-11T14:30:12,176 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_biology.yaml' 2026-05-11T14:30:12,177 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_chemistry.yaml' 2026-05-11T14:30:12,178 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_computer_science.yaml' 2026-05-11T14:30:12,179 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_european_history.yaml' 2026-05-11T14:30:12,180 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_geography.yaml' 2026-05-11T14:30:12,181 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_government_and_politics.yaml' 2026-05-11T14:30:12,183 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_macroeconomics.yaml' 2026-05-11T14:30:12,184 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_mathematics.yaml' 2026-05-11T14:30:12,185 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_microeconomics.yaml' 2026-05-11T14:30:12,186 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_physics.yaml' 2026-05-11T14:30:12,187 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_psychology.yaml' 2026-05-11T14:30:12,188 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_statistics.yaml' 2026-05-11T14:30:12,190 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_us_history.yaml' 2026-05-11T14:30:12,191 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_world_history.yaml' 2026-05-11T14:30:12,192 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_aging.yaml' 2026-05-11T14:30:12,193 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_sexuality.yaml' 2026-05-11T14:30:12,194 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_international_law.yaml' 2026-05-11T14:30:12,195 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_jurisprudence.yaml' 2026-05-11T14:30:12,196 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_logical_fallacies.yaml' 2026-05-11T14:30:12,198 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_machine_learning.yaml' 2026-05-11T14:30:12,199 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_management.yaml' 2026-05-11T14:30:12,200 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_marketing.yaml' 2026-05-11T14:30:12,201 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_medical_genetics.yaml' 2026-05-11T14:30:12,202 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_miscellaneous.yaml' 2026-05-11T14:30:12,203 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_disputes.yaml' 2026-05-11T14:30:12,204 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_scenarios.yaml' 2026-05-11T14:30:12,206 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_nutrition.yaml' 2026-05-11T14:30:12,207 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_philosophy.yaml' 2026-05-11T14:30:12,208 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_prehistory.yaml' 2026-05-11T14:30:12,209 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_accounting.yaml' 2026-05-11T14:30:12,210 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_law.yaml' 2026-05-11T14:30:12,211 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_medicine.yaml' 2026-05-11T14:30:12,213 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_psychology.yaml' 2026-05-11T14:30:12,214 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_public_relations.yaml' 2026-05-11T14:30:12,215 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_security_studies.yaml' 2026-05-11T14:30:12,216 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_sociology.yaml' 2026-05-11T14:30:12,217 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_us_foreign_policy.yaml' 2026-05-11T14:30:12,218 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_virology.yaml' 2026-05-11T14:30:12,219 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_world_religions.yaml' 2026-05-11T14:30:12,220 adding 'lm_eval/tasks/global_mmlu/full/yo/utils.py' 2026-05-11T14:30:12,223 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh.yaml' 2026-05-11T14:30:12,224 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_humanities.yaml' 2026-05-11T14:30:12,225 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_other.yaml' 2026-05-11T14:30:12,226 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_social_sciences.yaml' 2026-05-11T14:30:12,228 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_stem.yaml' 2026-05-11T14:30:12,229 adding 'lm_eval/tasks/global_mmlu/full/zh/_zh_template_yaml' 2026-05-11T14:30:12,230 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_abstract_algebra.yaml' 2026-05-11T14:30:12,231 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_anatomy.yaml' 2026-05-11T14:30:12,232 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_astronomy.yaml' 2026-05-11T14:30:12,233 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_business_ethics.yaml' 2026-05-11T14:30:12,234 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_clinical_knowledge.yaml' 2026-05-11T14:30:12,235 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_biology.yaml' 2026-05-11T14:30:12,237 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_chemistry.yaml' 2026-05-11T14:30:12,238 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_computer_science.yaml' 2026-05-11T14:30:12,239 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_mathematics.yaml' 2026-05-11T14:30:12,240 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_medicine.yaml' 2026-05-11T14:30:12,241 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_physics.yaml' 2026-05-11T14:30:12,242 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_computer_security.yaml' 2026-05-11T14:30:12,243 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_conceptual_physics.yaml' 2026-05-11T14:30:12,244 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_econometrics.yaml' 2026-05-11T14:30:12,246 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_electrical_engineering.yaml' 2026-05-11T14:30:12,247 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_elementary_mathematics.yaml' 2026-05-11T14:30:12,248 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_formal_logic.yaml' 2026-05-11T14:30:12,249 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_global_facts.yaml' 2026-05-11T14:30:12,250 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_biology.yaml' 2026-05-11T14:30:12,251 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_chemistry.yaml' 2026-05-11T14:30:12,253 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_computer_science.yaml' 2026-05-11T14:30:12,254 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_european_history.yaml' 2026-05-11T14:30:12,255 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_geography.yaml' 2026-05-11T14:30:12,256 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_government_and_politics.yaml' 2026-05-11T14:30:12,257 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_macroeconomics.yaml' 2026-05-11T14:30:12,258 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_mathematics.yaml' 2026-05-11T14:30:12,259 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_microeconomics.yaml' 2026-05-11T14:30:12,260 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_physics.yaml' 2026-05-11T14:30:12,261 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_psychology.yaml' 2026-05-11T14:30:12,262 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_statistics.yaml' 2026-05-11T14:30:12,263 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_us_history.yaml' 2026-05-11T14:30:12,264 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_world_history.yaml' 2026-05-11T14:30:12,266 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_aging.yaml' 2026-05-11T14:30:12,267 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_sexuality.yaml' 2026-05-11T14:30:12,268 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_international_law.yaml' 2026-05-11T14:30:12,269 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_jurisprudence.yaml' 2026-05-11T14:30:12,270 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_logical_fallacies.yaml' 2026-05-11T14:30:12,271 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_machine_learning.yaml' 2026-05-11T14:30:12,272 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_management.yaml' 2026-05-11T14:30:12,273 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_marketing.yaml' 2026-05-11T14:30:12,274 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_medical_genetics.yaml' 2026-05-11T14:30:12,276 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_miscellaneous.yaml' 2026-05-11T14:30:12,277 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_disputes.yaml' 2026-05-11T14:30:12,278 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_scenarios.yaml' 2026-05-11T14:30:12,279 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_nutrition.yaml' 2026-05-11T14:30:12,280 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_philosophy.yaml' 2026-05-11T14:30:12,282 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_prehistory.yaml' 2026-05-11T14:30:12,283 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_accounting.yaml' 2026-05-11T14:30:12,284 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_law.yaml' 2026-05-11T14:30:12,285 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_medicine.yaml' 2026-05-11T14:30:12,286 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_psychology.yaml' 2026-05-11T14:30:12,287 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_public_relations.yaml' 2026-05-11T14:30:12,289 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_security_studies.yaml' 2026-05-11T14:30:12,290 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_sociology.yaml' 2026-05-11T14:30:12,291 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_us_foreign_policy.yaml' 2026-05-11T14:30:12,292 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_virology.yaml' 2026-05-11T14:30:12,293 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_world_religions.yaml' 2026-05-11T14:30:12,295 adding 'lm_eval/tasks/global_mmlu/full/zh/utils.py' 2026-05-11T14:30:12,297 adding 'lm_eval/tasks/global_piqa/README.md' 2026-05-11T14:30:12,300 adding 'lm_eval/tasks/global_piqa/completions/_generate_config.py' 2026-05-11T14:30:12,302 adding 'lm_eval/tasks/global_piqa/completions/_global_piqa.yaml' 2026-05-11T14:30:12,303 adding 'lm_eval/tasks/global_piqa/completions/_template' 2026-05-11T14:30:12,304 adding 'lm_eval/tasks/global_piqa/completions/acm_arab.yaml' 2026-05-11T14:30:12,305 adding 'lm_eval/tasks/global_piqa/completions/acq_arab.yaml' 2026-05-11T14:30:12,306 adding 'lm_eval/tasks/global_piqa/completions/aeb_arab.yaml' 2026-05-11T14:30:12,307 adding 'lm_eval/tasks/global_piqa/completions/afb_arab.yaml' 2026-05-11T14:30:12,308 adding 'lm_eval/tasks/global_piqa/completions/als_latn.yaml' 2026-05-11T14:30:12,309 adding 'lm_eval/tasks/global_piqa/completions/amh_ethi.yaml' 2026-05-11T14:30:12,310 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_jord.yaml' 2026-05-11T14:30:12,311 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_leba.yaml' 2026-05-11T14:30:12,313 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_pale.yaml' 2026-05-11T14:30:12,314 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_syri.yaml' 2026-05-11T14:30:12,315 adding 'lm_eval/tasks/global_piqa/completions/arb_arab.yaml' 2026-05-11T14:30:12,316 adding 'lm_eval/tasks/global_piqa/completions/arq_arab.yaml' 2026-05-11T14:30:12,317 adding 'lm_eval/tasks/global_piqa/completions/ars_arab.yaml' 2026-05-11T14:30:12,318 adding 'lm_eval/tasks/global_piqa/completions/ary_arab.yaml' 2026-05-11T14:30:12,319 adding 'lm_eval/tasks/global_piqa/completions/arz_arab.yaml' 2026-05-11T14:30:12,320 adding 'lm_eval/tasks/global_piqa/completions/asm_beng.yaml' 2026-05-11T14:30:12,321 adding 'lm_eval/tasks/global_piqa/completions/azj_latn.yaml' 2026-05-11T14:30:12,322 adding 'lm_eval/tasks/global_piqa/completions/bam_latn.yaml' 2026-05-11T14:30:12,323 adding 'lm_eval/tasks/global_piqa/completions/bel_cyrl.yaml' 2026-05-11T14:30:12,324 adding 'lm_eval/tasks/global_piqa/completions/ben_beng.yaml' 2026-05-11T14:30:12,326 adding 'lm_eval/tasks/global_piqa/completions/ben_latn.yaml' 2026-05-11T14:30:12,327 adding 'lm_eval/tasks/global_piqa/completions/bho_deva.yaml' 2026-05-11T14:30:12,328 adding 'lm_eval/tasks/global_piqa/completions/bos_latn.yaml' 2026-05-11T14:30:12,329 adding 'lm_eval/tasks/global_piqa/completions/bsk_arab.yaml' 2026-05-11T14:30:12,330 adding 'lm_eval/tasks/global_piqa/completions/bul_cyrl.yaml' 2026-05-11T14:30:12,331 adding 'lm_eval/tasks/global_piqa/completions/cat_latn.yaml' 2026-05-11T14:30:12,332 adding 'lm_eval/tasks/global_piqa/completions/ces_latn.yaml' 2026-05-11T14:30:12,333 adding 'lm_eval/tasks/global_piqa/completions/ckb_arab.yaml' 2026-05-11T14:30:12,334 adding 'lm_eval/tasks/global_piqa/completions/ckm_latn.yaml' 2026-05-11T14:30:12,336 adding 'lm_eval/tasks/global_piqa/completions/cmn_hans.yaml' 2026-05-11T14:30:12,337 adding 'lm_eval/tasks/global_piqa/completions/cmn_hant.yaml' 2026-05-11T14:30:12,338 adding 'lm_eval/tasks/global_piqa/completions/deu_latn.yaml' 2026-05-11T14:30:12,339 adding 'lm_eval/tasks/global_piqa/completions/dhd_deva.yaml' 2026-05-11T14:30:12,340 adding 'lm_eval/tasks/global_piqa/completions/ekk_latn.yaml' 2026-05-11T14:30:12,341 adding 'lm_eval/tasks/global_piqa/completions/ekp_latn.yaml' 2026-05-11T14:30:12,342 adding 'lm_eval/tasks/global_piqa/completions/ell_grek.yaml' 2026-05-11T14:30:12,343 adding 'lm_eval/tasks/global_piqa/completions/eng_latn.yaml' 2026-05-11T14:30:12,345 adding 'lm_eval/tasks/global_piqa/completions/fao_latn.yaml' 2026-05-11T14:30:12,346 adding 'lm_eval/tasks/global_piqa/completions/fin_latn.yaml' 2026-05-11T14:30:12,347 adding 'lm_eval/tasks/global_piqa/completions/fra_latn_cana.yaml' 2026-05-11T14:30:12,348 adding 'lm_eval/tasks/global_piqa/completions/fra_latn_fran.yaml' 2026-05-11T14:30:12,349 adding 'lm_eval/tasks/global_piqa/completions/glg_latn.yaml' 2026-05-11T14:30:12,350 adding 'lm_eval/tasks/global_piqa/completions/guj_gujr.yaml' 2026-05-11T14:30:12,351 adding 'lm_eval/tasks/global_piqa/completions/hau_latn.yaml' 2026-05-11T14:30:12,352 adding 'lm_eval/tasks/global_piqa/completions/haw_latn.yaml' 2026-05-11T14:30:12,354 adding 'lm_eval/tasks/global_piqa/completions/heb_hebr.yaml' 2026-05-11T14:30:12,355 adding 'lm_eval/tasks/global_piqa/completions/hin_deva.yaml' 2026-05-11T14:30:12,356 adding 'lm_eval/tasks/global_piqa/completions/hrv_latn.yaml' 2026-05-11T14:30:12,357 adding 'lm_eval/tasks/global_piqa/completions/hun_latn.yaml' 2026-05-11T14:30:12,358 adding 'lm_eval/tasks/global_piqa/completions/hye_armn.yaml' 2026-05-11T14:30:12,359 adding 'lm_eval/tasks/global_piqa/completions/ibo_latn.yaml' 2026-05-11T14:30:12,360 adding 'lm_eval/tasks/global_piqa/completions/idu_latn.yaml' 2026-05-11T14:30:12,361 adding 'lm_eval/tasks/global_piqa/completions/ind_latn.yaml' 2026-05-11T14:30:12,362 adding 'lm_eval/tasks/global_piqa/completions/isl_latn.yaml' 2026-05-11T14:30:12,363 adding 'lm_eval/tasks/global_piqa/completions/iso_latn.yaml' 2026-05-11T14:30:12,364 adding 'lm_eval/tasks/global_piqa/completions/ita_latn.yaml' 2026-05-11T14:30:12,365 adding 'lm_eval/tasks/global_piqa/completions/jav_latn.yaml' 2026-05-11T14:30:12,366 adding 'lm_eval/tasks/global_piqa/completions/jpn_jpan.yaml' 2026-05-11T14:30:12,367 adding 'lm_eval/tasks/global_piqa/completions/kan_knda.yaml' 2026-05-11T14:30:12,368 adding 'lm_eval/tasks/global_piqa/completions/kat_geor.yaml' 2026-05-11T14:30:12,370 adding 'lm_eval/tasks/global_piqa/completions/kaz_cyrl.yaml' 2026-05-11T14:30:12,371 adding 'lm_eval/tasks/global_piqa/completions/kin_latn.yaml' 2026-05-11T14:30:12,372 adding 'lm_eval/tasks/global_piqa/completions/kir_cyrl.yaml' 2026-05-11T14:30:12,373 adding 'lm_eval/tasks/global_piqa/completions/kor_hang.yaml' 2026-05-11T14:30:12,374 adding 'lm_eval/tasks/global_piqa/completions/lin_latn.yaml' 2026-05-11T14:30:12,375 adding 'lm_eval/tasks/global_piqa/completions/lit_latn.yaml' 2026-05-11T14:30:12,376 adding 'lm_eval/tasks/global_piqa/completions/luo_latn.yaml' 2026-05-11T14:30:12,377 adding 'lm_eval/tasks/global_piqa/completions/mal_mlym.yaml' 2026-05-11T14:30:12,378 adding 'lm_eval/tasks/global_piqa/completions/mar_deva.yaml' 2026-05-11T14:30:12,380 adding 'lm_eval/tasks/global_piqa/completions/mkd_cyrl.yaml' 2026-05-11T14:30:12,381 adding 'lm_eval/tasks/global_piqa/completions/mni_beng.yaml' 2026-05-11T14:30:12,382 adding 'lm_eval/tasks/global_piqa/completions/mni_mtei.yaml' 2026-05-11T14:30:12,383 adding 'lm_eval/tasks/global_piqa/completions/nag_latn.yaml' 2026-05-11T14:30:12,384 adding 'lm_eval/tasks/global_piqa/completions/nld_latn.yaml' 2026-05-11T14:30:12,385 adding 'lm_eval/tasks/global_piqa/completions/nno_latn.yaml' 2026-05-11T14:30:12,386 adding 'lm_eval/tasks/global_piqa/completions/nob_latn.yaml' 2026-05-11T14:30:12,388 adding 'lm_eval/tasks/global_piqa/completions/npi_deva.yaml' 2026-05-11T14:30:12,389 adding 'lm_eval/tasks/global_piqa/completions/pan_guru.yaml' 2026-05-11T14:30:12,390 adding 'lm_eval/tasks/global_piqa/completions/pcm_latn.yaml' 2026-05-11T14:30:12,391 adding 'lm_eval/tasks/global_piqa/completions/pes_arab.yaml' 2026-05-11T14:30:12,392 adding 'lm_eval/tasks/global_piqa/completions/pol_latn.yaml' 2026-05-11T14:30:12,393 adding 'lm_eval/tasks/global_piqa/completions/por_latn_braz.yaml' 2026-05-11T14:30:12,394 adding 'lm_eval/tasks/global_piqa/completions/por_latn_port.yaml' 2026-05-11T14:30:12,395 adding 'lm_eval/tasks/global_piqa/completions/ron_latn.yaml' 2026-05-11T14:30:12,397 adding 'lm_eval/tasks/global_piqa/completions/rus_cyrl.yaml' 2026-05-11T14:30:12,398 adding 'lm_eval/tasks/global_piqa/completions/rwr_deva.yaml' 2026-05-11T14:30:12,399 adding 'lm_eval/tasks/global_piqa/completions/sin_sinh.yaml' 2026-05-11T14:30:12,400 adding 'lm_eval/tasks/global_piqa/completions/slk_latn.yaml' 2026-05-11T14:30:12,401 adding 'lm_eval/tasks/global_piqa/completions/slk_latn_sari.yaml' 2026-05-11T14:30:12,403 adding 'lm_eval/tasks/global_piqa/completions/slv_latn.yaml' 2026-05-11T14:30:12,404 adding 'lm_eval/tasks/global_piqa/completions/slv_latn_cerk.yaml' 2026-05-11T14:30:12,405 adding 'lm_eval/tasks/global_piqa/completions/snd_arab.yaml' 2026-05-11T14:30:12,406 adding 'lm_eval/tasks/global_piqa/completions/snd_deva.yaml' 2026-05-11T14:30:12,407 adding 'lm_eval/tasks/global_piqa/completions/spa_latn_mexi.yaml' 2026-05-11T14:30:12,409 adding 'lm_eval/tasks/global_piqa/completions/spa_latn_peru.yaml' 2026-05-11T14:30:12,410 adding 'lm_eval/tasks/global_piqa/completions/spa_latn_spai.yaml' 2026-05-11T14:30:12,411 adding 'lm_eval/tasks/global_piqa/completions/srp_cyrl.yaml' 2026-05-11T14:30:12,412 adding 'lm_eval/tasks/global_piqa/completions/srp_latn.yaml' 2026-05-11T14:30:12,413 adding 'lm_eval/tasks/global_piqa/completions/swe_latn.yaml' 2026-05-11T14:30:12,414 adding 'lm_eval/tasks/global_piqa/completions/swh_latn.yaml' 2026-05-11T14:30:12,415 adding 'lm_eval/tasks/global_piqa/completions/tam_taml.yaml' 2026-05-11T14:30:12,416 adding 'lm_eval/tasks/global_piqa/completions/tel_telu.yaml' 2026-05-11T14:30:12,418 adding 'lm_eval/tasks/global_piqa/completions/tgl_latn.yaml' 2026-05-11T14:30:12,419 adding 'lm_eval/tasks/global_piqa/completions/tha_thai.yaml' 2026-05-11T14:30:12,420 adding 'lm_eval/tasks/global_piqa/completions/tur_latn.yaml' 2026-05-11T14:30:12,421 adding 'lm_eval/tasks/global_piqa/completions/uig_arab.yaml' 2026-05-11T14:30:12,422 adding 'lm_eval/tasks/global_piqa/completions/ukr_cyrl.yaml' 2026-05-11T14:30:12,423 adding 'lm_eval/tasks/global_piqa/completions/urd_arab.yaml' 2026-05-11T14:30:12,424 adding 'lm_eval/tasks/global_piqa/completions/urd_latn.yaml' 2026-05-11T14:30:12,426 adding 'lm_eval/tasks/global_piqa/completions/urh_latn.yaml' 2026-05-11T14:30:12,427 adding 'lm_eval/tasks/global_piqa/completions/uzn_latn.yaml' 2026-05-11T14:30:12,428 adding 'lm_eval/tasks/global_piqa/completions/vie_latn.yaml' 2026-05-11T14:30:12,429 adding 'lm_eval/tasks/global_piqa/completions/yor_latn.yaml' 2026-05-11T14:30:12,430 adding 'lm_eval/tasks/global_piqa/completions/yue_hant.yaml' 2026-05-11T14:30:12,431 adding 'lm_eval/tasks/global_piqa/completions/zsm_latn.yaml' 2026-05-11T14:30:12,432 adding 'lm_eval/tasks/global_piqa/completions/zul_latn.yaml' 2026-05-11T14:30:12,436 adding 'lm_eval/tasks/global_piqa/prompted/_generate_config.py' 2026-05-11T14:30:12,438 adding 'lm_eval/tasks/global_piqa/prompted/_global_piqa.yaml' 2026-05-11T14:30:12,439 adding 'lm_eval/tasks/global_piqa/prompted/_template' 2026-05-11T14:30:12,441 adding 'lm_eval/tasks/global_piqa/prompted/acm_arab.yaml' 2026-05-11T14:30:12,442 adding 'lm_eval/tasks/global_piqa/prompted/acq_arab.yaml' 2026-05-11T14:30:12,443 adding 'lm_eval/tasks/global_piqa/prompted/aeb_arab.yaml' 2026-05-11T14:30:12,444 adding 'lm_eval/tasks/global_piqa/prompted/afb_arab.yaml' 2026-05-11T14:30:12,445 adding 'lm_eval/tasks/global_piqa/prompted/als_latn.yaml' 2026-05-11T14:30:12,446 adding 'lm_eval/tasks/global_piqa/prompted/amh_ethi.yaml' 2026-05-11T14:30:12,447 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_jord.yaml' 2026-05-11T14:30:12,449 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_leba.yaml' 2026-05-11T14:30:12,450 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_pale.yaml' 2026-05-11T14:30:12,451 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_syri.yaml' 2026-05-11T14:30:12,452 adding 'lm_eval/tasks/global_piqa/prompted/arb_arab.yaml' 2026-05-11T14:30:12,453 adding 'lm_eval/tasks/global_piqa/prompted/arq_arab.yaml' 2026-05-11T14:30:12,454 adding 'lm_eval/tasks/global_piqa/prompted/ars_arab.yaml' 2026-05-11T14:30:12,456 adding 'lm_eval/tasks/global_piqa/prompted/ary_arab.yaml' 2026-05-11T14:30:12,457 adding 'lm_eval/tasks/global_piqa/prompted/arz_arab.yaml' 2026-05-11T14:30:12,458 adding 'lm_eval/tasks/global_piqa/prompted/asm_beng.yaml' 2026-05-11T14:30:12,459 adding 'lm_eval/tasks/global_piqa/prompted/azj_latn.yaml' 2026-05-11T14:30:12,460 adding 'lm_eval/tasks/global_piqa/prompted/bam_latn.yaml' 2026-05-11T14:30:12,461 adding 'lm_eval/tasks/global_piqa/prompted/bel_cyrl.yaml' 2026-05-11T14:30:12,463 adding 'lm_eval/tasks/global_piqa/prompted/ben_beng.yaml' 2026-05-11T14:30:12,464 adding 'lm_eval/tasks/global_piqa/prompted/ben_latn.yaml' 2026-05-11T14:30:12,465 adding 'lm_eval/tasks/global_piqa/prompted/bho_deva.yaml' 2026-05-11T14:30:12,466 adding 'lm_eval/tasks/global_piqa/prompted/bos_latn.yaml' 2026-05-11T14:30:12,467 adding 'lm_eval/tasks/global_piqa/prompted/bsk_arab.yaml' 2026-05-11T14:30:12,468 adding 'lm_eval/tasks/global_piqa/prompted/bul_cyrl.yaml' 2026-05-11T14:30:12,469 adding 'lm_eval/tasks/global_piqa/prompted/cat_latn.yaml' 2026-05-11T14:30:12,470 adding 'lm_eval/tasks/global_piqa/prompted/ces_latn.yaml' 2026-05-11T14:30:12,472 adding 'lm_eval/tasks/global_piqa/prompted/ckb_arab.yaml' 2026-05-11T14:30:12,473 adding 'lm_eval/tasks/global_piqa/prompted/ckm_latn.yaml' 2026-05-11T14:30:12,474 adding 'lm_eval/tasks/global_piqa/prompted/cmn_hans.yaml' 2026-05-11T14:30:12,475 adding 'lm_eval/tasks/global_piqa/prompted/cmn_hant.yaml' 2026-05-11T14:30:12,476 adding 'lm_eval/tasks/global_piqa/prompted/deu_latn.yaml' 2026-05-11T14:30:12,477 adding 'lm_eval/tasks/global_piqa/prompted/dhd_deva.yaml' 2026-05-11T14:30:12,478 adding 'lm_eval/tasks/global_piqa/prompted/ekk_latn.yaml' 2026-05-11T14:30:12,480 adding 'lm_eval/tasks/global_piqa/prompted/ekp_latn.yaml' 2026-05-11T14:30:12,481 adding 'lm_eval/tasks/global_piqa/prompted/ell_grek.yaml' 2026-05-11T14:30:12,482 adding 'lm_eval/tasks/global_piqa/prompted/eng_latn.yaml' 2026-05-11T14:30:12,483 adding 'lm_eval/tasks/global_piqa/prompted/fao_latn.yaml' 2026-05-11T14:30:12,484 adding 'lm_eval/tasks/global_piqa/prompted/fin_latn.yaml' 2026-05-11T14:30:12,485 adding 'lm_eval/tasks/global_piqa/prompted/fra_latn_cana.yaml' 2026-05-11T14:30:12,487 adding 'lm_eval/tasks/global_piqa/prompted/fra_latn_fran.yaml' 2026-05-11T14:30:12,488 adding 'lm_eval/tasks/global_piqa/prompted/glg_latn.yaml' 2026-05-11T14:30:12,489 adding 'lm_eval/tasks/global_piqa/prompted/guj_gujr.yaml' 2026-05-11T14:30:12,490 adding 'lm_eval/tasks/global_piqa/prompted/hau_latn.yaml' 2026-05-11T14:30:12,491 adding 'lm_eval/tasks/global_piqa/prompted/haw_latn.yaml' 2026-05-11T14:30:12,492 adding 'lm_eval/tasks/global_piqa/prompted/heb_hebr.yaml' 2026-05-11T14:30:12,493 adding 'lm_eval/tasks/global_piqa/prompted/hin_deva.yaml' 2026-05-11T14:30:12,494 adding 'lm_eval/tasks/global_piqa/prompted/hrv_latn.yaml' 2026-05-11T14:30:12,495 adding 'lm_eval/tasks/global_piqa/prompted/hun_latn.yaml' 2026-05-11T14:30:12,497 adding 'lm_eval/tasks/global_piqa/prompted/hye_armn.yaml' 2026-05-11T14:30:12,498 adding 'lm_eval/tasks/global_piqa/prompted/ibo_latn.yaml' 2026-05-11T14:30:12,499 adding 'lm_eval/tasks/global_piqa/prompted/idu_latn.yaml' 2026-05-11T14:30:12,500 adding 'lm_eval/tasks/global_piqa/prompted/ind_latn.yaml' 2026-05-11T14:30:12,501 adding 'lm_eval/tasks/global_piqa/prompted/isl_latn.yaml' 2026-05-11T14:30:12,503 adding 'lm_eval/tasks/global_piqa/prompted/iso_latn.yaml' 2026-05-11T14:30:12,504 adding 'lm_eval/tasks/global_piqa/prompted/ita_latn.yaml' 2026-05-11T14:30:12,505 adding 'lm_eval/tasks/global_piqa/prompted/jav_latn.yaml' 2026-05-11T14:30:12,506 adding 'lm_eval/tasks/global_piqa/prompted/jpn_jpan.yaml' 2026-05-11T14:30:12,507 adding 'lm_eval/tasks/global_piqa/prompted/kan_knda.yaml' 2026-05-11T14:30:12,508 adding 'lm_eval/tasks/global_piqa/prompted/kat_geor.yaml' 2026-05-11T14:30:12,510 adding 'lm_eval/tasks/global_piqa/prompted/kaz_cyrl.yaml' 2026-05-11T14:30:12,511 adding 'lm_eval/tasks/global_piqa/prompted/kin_latn.yaml' 2026-05-11T14:30:12,512 adding 'lm_eval/tasks/global_piqa/prompted/kir_cyrl.yaml' 2026-05-11T14:30:12,513 adding 'lm_eval/tasks/global_piqa/prompted/kor_hang.yaml' 2026-05-11T14:30:12,514 adding 'lm_eval/tasks/global_piqa/prompted/lin_latn.yaml' 2026-05-11T14:30:12,515 adding 'lm_eval/tasks/global_piqa/prompted/lit_latn.yaml' 2026-05-11T14:30:12,516 adding 'lm_eval/tasks/global_piqa/prompted/luo_latn.yaml' 2026-05-11T14:30:12,518 adding 'lm_eval/tasks/global_piqa/prompted/mal_mlym.yaml' 2026-05-11T14:30:12,519 adding 'lm_eval/tasks/global_piqa/prompted/mar_deva.yaml' 2026-05-11T14:30:12,520 adding 'lm_eval/tasks/global_piqa/prompted/mkd_cyrl.yaml' 2026-05-11T14:30:12,521 adding 'lm_eval/tasks/global_piqa/prompted/mni_beng.yaml' 2026-05-11T14:30:12,522 adding 'lm_eval/tasks/global_piqa/prompted/mni_mtei.yaml' 2026-05-11T14:30:12,523 adding 'lm_eval/tasks/global_piqa/prompted/nag_latn.yaml' 2026-05-11T14:30:12,524 adding 'lm_eval/tasks/global_piqa/prompted/nld_latn.yaml' 2026-05-11T14:30:12,525 adding 'lm_eval/tasks/global_piqa/prompted/nno_latn.yaml' 2026-05-11T14:30:12,527 adding 'lm_eval/tasks/global_piqa/prompted/nob_latn.yaml' 2026-05-11T14:30:12,528 adding 'lm_eval/tasks/global_piqa/prompted/npi_deva.yaml' 2026-05-11T14:30:12,529 adding 'lm_eval/tasks/global_piqa/prompted/pan_guru.yaml' 2026-05-11T14:30:12,530 adding 'lm_eval/tasks/global_piqa/prompted/pcm_latn.yaml' 2026-05-11T14:30:12,531 adding 'lm_eval/tasks/global_piqa/prompted/pes_arab.yaml' 2026-05-11T14:30:12,532 adding 'lm_eval/tasks/global_piqa/prompted/pol_latn.yaml' 2026-05-11T14:30:12,534 adding 'lm_eval/tasks/global_piqa/prompted/por_latn_braz.yaml' 2026-05-11T14:30:12,535 adding 'lm_eval/tasks/global_piqa/prompted/por_latn_port.yaml' 2026-05-11T14:30:12,536 adding 'lm_eval/tasks/global_piqa/prompted/ron_latn.yaml' 2026-05-11T14:30:12,537 adding 'lm_eval/tasks/global_piqa/prompted/rus_cyrl.yaml' 2026-05-11T14:30:12,538 adding 'lm_eval/tasks/global_piqa/prompted/rwr_deva.yaml' 2026-05-11T14:30:12,539 adding 'lm_eval/tasks/global_piqa/prompted/sin_sinh.yaml' 2026-05-11T14:30:12,540 adding 'lm_eval/tasks/global_piqa/prompted/slk_latn.yaml' 2026-05-11T14:30:12,541 adding 'lm_eval/tasks/global_piqa/prompted/slk_latn_sari.yaml' 2026-05-11T14:30:12,543 adding 'lm_eval/tasks/global_piqa/prompted/slv_latn.yaml' 2026-05-11T14:30:12,544 adding 'lm_eval/tasks/global_piqa/prompted/slv_latn_cerk.yaml' 2026-05-11T14:30:12,545 adding 'lm_eval/tasks/global_piqa/prompted/snd_arab.yaml' 2026-05-11T14:30:12,546 adding 'lm_eval/tasks/global_piqa/prompted/snd_deva.yaml' 2026-05-11T14:30:12,547 adding 'lm_eval/tasks/global_piqa/prompted/spa_latn_mexi.yaml' 2026-05-11T14:30:12,548 adding 'lm_eval/tasks/global_piqa/prompted/spa_latn_peru.yaml' 2026-05-11T14:30:12,549 adding 'lm_eval/tasks/global_piqa/prompted/spa_latn_spai.yaml' 2026-05-11T14:30:12,550 adding 'lm_eval/tasks/global_piqa/prompted/srp_cyrl.yaml' 2026-05-11T14:30:12,551 adding 'lm_eval/tasks/global_piqa/prompted/srp_latn.yaml' 2026-05-11T14:30:12,553 adding 'lm_eval/tasks/global_piqa/prompted/swe_latn.yaml' 2026-05-11T14:30:12,554 adding 'lm_eval/tasks/global_piqa/prompted/swh_latn.yaml' 2026-05-11T14:30:12,555 adding 'lm_eval/tasks/global_piqa/prompted/tam_taml.yaml' 2026-05-11T14:30:12,556 adding 'lm_eval/tasks/global_piqa/prompted/tel_telu.yaml' 2026-05-11T14:30:12,557 adding 'lm_eval/tasks/global_piqa/prompted/tgl_latn.yaml' 2026-05-11T14:30:12,558 adding 'lm_eval/tasks/global_piqa/prompted/tha_thai.yaml' 2026-05-11T14:30:12,559 adding 'lm_eval/tasks/global_piqa/prompted/tur_latn.yaml' 2026-05-11T14:30:12,561 adding 'lm_eval/tasks/global_piqa/prompted/uig_arab.yaml' 2026-05-11T14:30:12,562 adding 'lm_eval/tasks/global_piqa/prompted/ukr_cyrl.yaml' 2026-05-11T14:30:12,563 adding 'lm_eval/tasks/global_piqa/prompted/urd_arab.yaml' 2026-05-11T14:30:12,564 adding 'lm_eval/tasks/global_piqa/prompted/urd_latn.yaml' 2026-05-11T14:30:12,565 adding 'lm_eval/tasks/global_piqa/prompted/urh_latn.yaml' 2026-05-11T14:30:12,566 adding 'lm_eval/tasks/global_piqa/prompted/uzn_latn.yaml' 2026-05-11T14:30:12,567 adding 'lm_eval/tasks/global_piqa/prompted/vie_latn.yaml' 2026-05-11T14:30:12,569 adding 'lm_eval/tasks/global_piqa/prompted/yor_latn.yaml' 2026-05-11T14:30:12,570 adding 'lm_eval/tasks/global_piqa/prompted/yue_hant.yaml' 2026-05-11T14:30:12,571 adding 'lm_eval/tasks/global_piqa/prompted/zsm_latn.yaml' 2026-05-11T14:30:12,572 adding 'lm_eval/tasks/global_piqa/prompted/zul_latn.yaml' 2026-05-11T14:30:12,574 adding 'lm_eval/tasks/glue/README.md' 2026-05-11T14:30:12,576 adding 'lm_eval/tasks/glue/cola/default.yaml' 2026-05-11T14:30:12,578 adding 'lm_eval/tasks/glue/mnli/default.yaml' 2026-05-11T14:30:12,579 adding 'lm_eval/tasks/glue/mnli/mismatch.yaml' 2026-05-11T14:30:12,580 adding 'lm_eval/tasks/glue/mnli/utils.py' 2026-05-11T14:30:12,582 adding 'lm_eval/tasks/glue/mrpc/default.yaml' 2026-05-11T14:30:12,583 adding 'lm_eval/tasks/glue/qnli/default.yaml' 2026-05-11T14:30:12,585 adding 'lm_eval/tasks/glue/qqp/default.yaml' 2026-05-11T14:30:12,586 adding 'lm_eval/tasks/glue/rte/default.yaml' 2026-05-11T14:30:12,588 adding 'lm_eval/tasks/glue/sst2/default.yaml' 2026-05-11T14:30:12,589 adding 'lm_eval/tasks/glue/wnli/default.yaml' 2026-05-11T14:30:12,591 adding 'lm_eval/tasks/gpqa/README.md' 2026-05-11T14:30:12,593 adding 'lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py' 2026-05-11T14:30:12,594 adding 'lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml' 2026-05-11T14:30:12,595 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml' 2026-05-11T14:30:12,596 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml' 2026-05-11T14:30:12,597 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml' 2026-05-11T14:30:12,598 adding 'lm_eval/tasks/gpqa/cot_n_shot/utils.py' 2026-05-11T14:30:12,600 adding 'lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py' 2026-05-11T14:30:12,601 adding 'lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml' 2026-05-11T14:30:12,603 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml' 2026-05-11T14:30:12,604 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml' 2026-05-11T14:30:12,606 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml' 2026-05-11T14:30:12,607 adding 'lm_eval/tasks/gpqa/cot_zeroshot/utils.py' 2026-05-11T14:30:12,610 adding 'lm_eval/tasks/gpqa/generative/_generate_configs.py' 2026-05-11T14:30:12,611 adding 'lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml' 2026-05-11T14:30:12,613 adding 'lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml' 2026-05-11T14:30:12,614 adding 'lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml' 2026-05-11T14:30:12,615 adding 'lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml' 2026-05-11T14:30:12,616 adding 'lm_eval/tasks/gpqa/generative/utils.py' 2026-05-11T14:30:12,618 adding 'lm_eval/tasks/gpqa/n_shot/_generate_configs.py' 2026-05-11T14:30:12,619 adding 'lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml' 2026-05-11T14:30:12,620 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml' 2026-05-11T14:30:12,622 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml' 2026-05-11T14:30:12,623 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml' 2026-05-11T14:30:12,624 adding 'lm_eval/tasks/gpqa/n_shot/utils.py' 2026-05-11T14:30:12,626 adding 'lm_eval/tasks/gpqa/zeroshot/_generate_configs.py' 2026-05-11T14:30:12,627 adding 'lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml' 2026-05-11T14:30:12,628 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml' 2026-05-11T14:30:12,629 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml' 2026-05-11T14:30:12,630 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml' 2026-05-11T14:30:12,632 adding 'lm_eval/tasks/gpqa/zeroshot/utils.py' 2026-05-11T14:30:12,634 adding 'lm_eval/tasks/graphwalks/README.md' 2026-05-11T14:30:12,635 adding 'lm_eval/tasks/graphwalks/graphwalks.yaml' 2026-05-11T14:30:12,636 adding 'lm_eval/tasks/graphwalks/graphwalks_128k.yaml' 2026-05-11T14:30:12,637 adding 'lm_eval/tasks/graphwalks/graphwalks_1M.yaml' 2026-05-11T14:30:12,639 adding 'lm_eval/tasks/graphwalks/utils.py' 2026-05-11T14:30:12,641 adding 'lm_eval/tasks/groundcocoa/README.md' 2026-05-11T14:30:12,642 adding 'lm_eval/tasks/groundcocoa/groundcocoa.yaml' 2026-05-11T14:30:12,644 adding 'lm_eval/tasks/groundcocoa/utils.py' 2026-05-11T14:30:12,646 adding 'lm_eval/tasks/gsm8k/README.md' 2026-05-11T14:30:12,648 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-llama.yaml' 2026-05-11T14:30:12,649 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml' 2026-05-11T14:30:12,651 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml' 2026-05-11T14:30:12,652 adding 'lm_eval/tasks/gsm8k/gsm8k-cot.yaml' 2026-05-11T14:30:12,654 adding 'lm_eval/tasks/gsm8k/gsm8k.yaml' 2026-05-11T14:30:12,656 adding 'lm_eval/tasks/gsm8k_platinum/README.md' 2026-05-11T14:30:12,657 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-llama.yaml' 2026-05-11T14:30:12,658 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-self-consistency.yaml' 2026-05-11T14:30:12,659 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-zeroshot.yaml' 2026-05-11T14:30:12,661 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot.yaml' 2026-05-11T14:30:12,662 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum.yaml' 2026-05-11T14:30:12,663 adding 'lm_eval/tasks/gsm_plus/README.md' 2026-05-11T14:30:12,665 adding 'lm_eval/tasks/gsm_plus/gsm_plus.yaml' 2026-05-11T14:30:12,666 adding 'lm_eval/tasks/gsm_plus/gsm_plus_mini.yaml' 2026-05-11T14:30:12,668 adding 'lm_eval/tasks/haerae/README.md' 2026-05-11T14:30:12,669 adding 'lm_eval/tasks/haerae/_default_haerae_yaml' 2026-05-11T14:30:12,670 adding 'lm_eval/tasks/haerae/_haerae.yaml' 2026-05-11T14:30:12,671 adding 'lm_eval/tasks/haerae/haerae_gk.yaml' 2026-05-11T14:30:12,672 adding 'lm_eval/tasks/haerae/haerae_hi.yaml' 2026-05-11T14:30:12,673 adding 'lm_eval/tasks/haerae/haerae_lw.yaml' 2026-05-11T14:30:12,674 adding 'lm_eval/tasks/haerae/haerae_rw.yaml' 2026-05-11T14:30:12,675 adding 'lm_eval/tasks/haerae/haerae_sn.yaml' 2026-05-11T14:30:12,677 adding 'lm_eval/tasks/headqa/README.md' 2026-05-11T14:30:12,678 adding 'lm_eval/tasks/headqa/headqa_en.yaml' 2026-05-11T14:30:12,679 adding 'lm_eval/tasks/headqa/headqa_es.yaml' 2026-05-11T14:30:12,681 adding 'lm_eval/tasks/hellaswag/README.md' 2026-05-11T14:30:12,683 adding 'lm_eval/tasks/hellaswag/hellaswag.yaml' 2026-05-11T14:30:12,684 adding 'lm_eval/tasks/hellaswag/utils.py' 2026-05-11T14:30:12,686 adding 'lm_eval/tasks/hendrycks_ethics/README.md' 2026-05-11T14:30:12,687 adding 'lm_eval/tasks/hendrycks_ethics/commonsense.yaml' 2026-05-11T14:30:12,688 adding 'lm_eval/tasks/hendrycks_ethics/deontology.yaml' 2026-05-11T14:30:12,689 adding 'lm_eval/tasks/hendrycks_ethics/justice.yaml' 2026-05-11T14:30:12,690 adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml' 2026-05-11T14:30:12,691 adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml' 2026-05-11T14:30:12,693 adding 'lm_eval/tasks/hendrycks_ethics/utils.py' 2026-05-11T14:30:12,694 adding 'lm_eval/tasks/hendrycks_ethics/virtue.yaml' 2026-05-11T14:30:12,696 adding 'lm_eval/tasks/hendrycks_math/README.md' 2026-05-11T14:30:12,697 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math.yaml' 2026-05-11T14:30:12,699 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math500.yaml' 2026-05-11T14:30:12,700 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml' 2026-05-11T14:30:12,701 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml' 2026-05-11T14:30:12,702 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml' 2026-05-11T14:30:12,703 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml' 2026-05-11T14:30:12,704 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml' 2026-05-11T14:30:12,706 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml' 2026-05-11T14:30:12,707 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml' 2026-05-11T14:30:12,709 adding 'lm_eval/tasks/hendrycks_math/utils.py' 2026-05-11T14:30:12,710 adding 'lm_eval/tasks/histoires_morales/README.md' 2026-05-11T14:30:12,712 adding 'lm_eval/tasks/histoires_morales/histoires_morales.yaml' 2026-05-11T14:30:12,713 adding 'lm_eval/tasks/histoires_morales/utils.py' 2026-05-11T14:30:12,715 adding 'lm_eval/tasks/hrm8k/README.md' 2026-05-11T14:30:12,717 adding 'lm_eval/tasks/hrm8k/default/_hrm8k_yaml' 2026-05-11T14:30:12,718 adding 'lm_eval/tasks/hrm8k/default/hrm8k.yaml' 2026-05-11T14:30:12,719 adding 'lm_eval/tasks/hrm8k/default/hrm8k_gsm8k.yaml' 2026-05-11T14:30:12,720 adding 'lm_eval/tasks/hrm8k/default/hrm8k_ksm.yaml' 2026-05-11T14:30:12,721 adding 'lm_eval/tasks/hrm8k/default/hrm8k_math.yaml' 2026-05-11T14:30:12,722 adding 'lm_eval/tasks/hrm8k/default/hrm8k_mmmlu.yaml' 2026-05-11T14:30:12,723 adding 'lm_eval/tasks/hrm8k/default/hrm8k_omni_math.yaml' 2026-05-11T14:30:12,725 adding 'lm_eval/tasks/hrm8k/default/utils.py' 2026-05-11T14:30:12,727 adding 'lm_eval/tasks/hrm8k/en/_hrm8k_en_yaml' 2026-05-11T14:30:12,728 adding 'lm_eval/tasks/hrm8k/en/hrm8k_en.yaml' 2026-05-11T14:30:12,729 adding 'lm_eval/tasks/hrm8k/en/hrm8k_gsm8k_en.yaml' 2026-05-11T14:30:12,730 adding 'lm_eval/tasks/hrm8k/en/hrm8k_ksm_en.yaml' 2026-05-11T14:30:12,731 adding 'lm_eval/tasks/hrm8k/en/hrm8k_math_en.yaml' 2026-05-11T14:30:12,732 adding 'lm_eval/tasks/hrm8k/en/hrm8k_mmmlu_en.yaml' 2026-05-11T14:30:12,733 adding 'lm_eval/tasks/hrm8k/en/hrm8k_omni_math_en.yaml' 2026-05-11T14:30:12,734 adding 'lm_eval/tasks/hrm8k/en/utils.py' 2026-05-11T14:30:12,737 adding 'lm_eval/tasks/humaneval/README.md' 2026-05-11T14:30:12,738 adding 'lm_eval/tasks/humaneval/humaneval.yaml' 2026-05-11T14:30:12,739 adding 'lm_eval/tasks/humaneval/humaneval_64.yaml' 2026-05-11T14:30:12,740 adding 'lm_eval/tasks/humaneval/humaneval_64_instruct.yaml' 2026-05-11T14:30:12,741 adding 'lm_eval/tasks/humaneval/humaneval_instruct.yaml' 2026-05-11T14:30:12,742 adding 'lm_eval/tasks/humaneval/humaneval_plus.yaml' 2026-05-11T14:30:12,744 adding 'lm_eval/tasks/humaneval/utils.py' 2026-05-11T14:30:12,745 adding 'lm_eval/tasks/humaneval_infilling/README.md' 2026-05-11T14:30:12,747 adding 'lm_eval/tasks/humaneval_infilling/humaneval_infilling.yaml' 2026-05-11T14:30:12,748 adding 'lm_eval/tasks/humaneval_infilling/multi_line_infilling.yaml' 2026-05-11T14:30:12,749 adding 'lm_eval/tasks/humaneval_infilling/random_span_infilling.yaml' 2026-05-11T14:30:12,750 adding 'lm_eval/tasks/humaneval_infilling/random_span_infilling_light.yaml' 2026-05-11T14:30:12,751 adding 'lm_eval/tasks/humaneval_infilling/single_line_infilling.yaml' 2026-05-11T14:30:12,752 adding 'lm_eval/tasks/humaneval_infilling/utils.py' 2026-05-11T14:30:12,754 adding 'lm_eval/tasks/icelandic_winogrande/README.md' 2026-05-11T14:30:12,756 adding 'lm_eval/tasks/icelandic_winogrande/default.yaml' 2026-05-11T14:30:12,757 adding 'lm_eval/tasks/icelandic_winogrande/preprocess_winogrande.py' 2026-05-11T14:30:12,759 adding 'lm_eval/tasks/ifeval/README.md' 2026-05-11T14:30:12,760 adding 'lm_eval/tasks/ifeval/ifeval.yaml' 2026-05-11T14:30:12,766 adding 'lm_eval/tasks/ifeval/instructions.py' 2026-05-11T14:30:12,768 adding 'lm_eval/tasks/ifeval/instructions_registry.py' 2026-05-11T14:30:12,771 adding 'lm_eval/tasks/ifeval/instructions_util.py' 2026-05-11T14:30:12,773 adding 'lm_eval/tasks/ifeval/utils.py' 2026-05-11T14:30:12,774 adding 'lm_eval/tasks/ifeval/multilingual/ifeval_ca.yaml' 2026-05-11T14:30:12,776 adding 'lm_eval/tasks/ifeval/multilingual/ifeval_es.yaml' 2026-05-11T14:30:12,777 adding 'lm_eval/tasks/ifeval/multilingual/instructions_registry.py' 2026-05-11T14:30:12,778 adding 'lm_eval/tasks/ifeval/multilingual/utils.py' 2026-05-11T14:30:12,780 adding 'lm_eval/tasks/ifeval/multilingual/instruction_utils/__init__.py' 2026-05-11T14:30:12,782 adding 'lm_eval/tasks/ifeval/multilingual/instruction_utils/ca_instructions_util.py' 2026-05-11T14:30:12,783 adding 'lm_eval/tasks/ifeval/multilingual/instruction_utils/es_instructions_util.py' 2026-05-11T14:30:12,785 adding 'lm_eval/tasks/ifeval/multilingual/instructions/__init__.py' 2026-05-11T14:30:12,791 adding 'lm_eval/tasks/ifeval/multilingual/instructions/ca_instructions.py' 2026-05-11T14:30:12,798 adding 'lm_eval/tasks/ifeval/multilingual/instructions/es_instructions.py' 2026-05-11T14:30:12,801 adding 'lm_eval/tasks/include/README.md' 2026-05-11T14:30:12,804 adding 'lm_eval/tasks/include/default/Albanian/_albanian_template_yaml' 2026-05-11T14:30:12,805 adding 'lm_eval/tasks/include/default/Albanian/_include_base_44_albanian.yaml' 2026-05-11T14:30:12,806 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_arts_humanities.yaml' 2026-05-11T14:30:12,807 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_business_commerce.yaml' 2026-05-11T14:30:12,808 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_health_oriented_education.yaml' 2026-05-11T14:30:12,809 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_social_science.yaml' 2026-05-11T14:30:12,810 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_stem.yaml' 2026-05-11T14:30:12,811 adding 'lm_eval/tasks/include/default/Albanian/utils.py' 2026-05-11T14:30:12,813 adding 'lm_eval/tasks/include/default/Arabic/_arabic_template_yaml' 2026-05-11T14:30:12,814 adding 'lm_eval/tasks/include/default/Arabic/_include_base_44_arabic.yaml' 2026-05-11T14:30:12,815 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_arts_humanities.yaml' 2026-05-11T14:30:12,816 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_business_commerce.yaml' 2026-05-11T14:30:12,817 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_driving_license.yaml' 2026-05-11T14:30:12,819 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_general_knowledge.yaml' 2026-05-11T14:30:12,820 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_social_science.yaml' 2026-05-11T14:30:12,821 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_stem.yaml' 2026-05-11T14:30:12,822 adding 'lm_eval/tasks/include/default/Arabic/utils.py' 2026-05-11T14:30:12,824 adding 'lm_eval/tasks/include/default/Armenian/_armenian_template_yaml' 2026-05-11T14:30:12,825 adding 'lm_eval/tasks/include/default/Armenian/_include_base_44_armenian.yaml' 2026-05-11T14:30:12,826 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_arts_humanities.yaml' 2026-05-11T14:30:12,827 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_driving_license.yaml' 2026-05-11T14:30:12,828 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_social_science.yaml' 2026-05-11T14:30:12,830 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_stem.yaml' 2026-05-11T14:30:12,831 adding 'lm_eval/tasks/include/default/Armenian/utils.py' 2026-05-11T14:30:12,833 adding 'lm_eval/tasks/include/default/Azerbaijani/_azerbaijani_template_yaml' 2026-05-11T14:30:12,834 adding 'lm_eval/tasks/include/default/Azerbaijani/_include_base_44_azerbaijani.yaml' 2026-05-11T14:30:12,835 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml' 2026-05-11T14:30:12,836 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml' 2026-05-11T14:30:12,837 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml' 2026-05-11T14:30:12,839 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml' 2026-05-11T14:30:12,840 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_social_science.yaml' 2026-05-11T14:30:12,841 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_stem.yaml' 2026-05-11T14:30:12,842 adding 'lm_eval/tasks/include/default/Azerbaijani/utils.py' 2026-05-11T14:30:12,844 adding 'lm_eval/tasks/include/default/Basque/_basque_template_yaml' 2026-05-11T14:30:12,845 adding 'lm_eval/tasks/include/default/Basque/_include_base_44_basque.yaml' 2026-05-11T14:30:12,846 adding 'lm_eval/tasks/include/default/Basque/include_base_44_basque_professional_certification.yaml' 2026-05-11T14:30:12,848 adding 'lm_eval/tasks/include/default/Basque/utils.py' 2026-05-11T14:30:12,849 adding 'lm_eval/tasks/include/default/Belarusian/_belarusian_template_yaml' 2026-05-11T14:30:12,851 adding 'lm_eval/tasks/include/default/Belarusian/_include_base_44_belarusian.yaml' 2026-05-11T14:30:12,852 adding 'lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_arts_humanities.yaml' 2026-05-11T14:30:12,853 adding 'lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_social_science.yaml' 2026-05-11T14:30:12,854 adding 'lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_stem.yaml' 2026-05-11T14:30:12,855 adding 'lm_eval/tasks/include/default/Belarusian/utils.py' 2026-05-11T14:30:12,857 adding 'lm_eval/tasks/include/default/Bengali/_bengali_template_yaml' 2026-05-11T14:30:12,858 adding 'lm_eval/tasks/include/default/Bengali/_include_base_44_bengali.yaml' 2026-05-11T14:30:12,859 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_arts_humanities.yaml' 2026-05-11T14:30:12,860 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_general_knowledge.yaml' 2026-05-11T14:30:12,861 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_professional_certification.yaml' 2026-05-11T14:30:12,862 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_stem.yaml' 2026-05-11T14:30:12,863 adding 'lm_eval/tasks/include/default/Bengali/utils.py' 2026-05-11T14:30:12,865 adding 'lm_eval/tasks/include/default/Bulgarian/_bulgarian_template_yaml' 2026-05-11T14:30:12,866 adding 'lm_eval/tasks/include/default/Bulgarian/_include_base_44_bulgarian.yaml' 2026-05-11T14:30:12,867 adding 'lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml' 2026-05-11T14:30:12,868 adding 'lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_social_science.yaml' 2026-05-11T14:30:12,870 adding 'lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_stem.yaml' 2026-05-11T14:30:12,871 adding 'lm_eval/tasks/include/default/Bulgarian/utils.py' 2026-05-11T14:30:12,873 adding 'lm_eval/tasks/include/default/Chinese/_chinese_template_yaml' 2026-05-11T14:30:12,874 adding 'lm_eval/tasks/include/default/Chinese/_include_base_44_chinese.yaml' 2026-05-11T14:30:12,875 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_applied_science.yaml' 2026-05-11T14:30:12,876 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_arts_humanities.yaml' 2026-05-11T14:30:12,877 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_business_commerce.yaml' 2026-05-11T14:30:12,879 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_driving_license.yaml' 2026-05-11T14:30:12,880 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_health_oriented_education.yaml' 2026-05-11T14:30:12,881 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_professional_certification.yaml' 2026-05-11T14:30:12,882 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_social_science.yaml' 2026-05-11T14:30:12,883 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_stem.yaml' 2026-05-11T14:30:12,884 adding 'lm_eval/tasks/include/default/Chinese/utils.py' 2026-05-11T14:30:12,886 adding 'lm_eval/tasks/include/default/Croatian/_croatian_template_yaml' 2026-05-11T14:30:12,887 adding 'lm_eval/tasks/include/default/Croatian/_include_base_44_croatian.yaml' 2026-05-11T14:30:12,889 adding 'lm_eval/tasks/include/default/Croatian/include_base_44_croatian_arts_humanities.yaml' 2026-05-11T14:30:12,890 adding 'lm_eval/tasks/include/default/Croatian/include_base_44_croatian_social_science.yaml' 2026-05-11T14:30:12,891 adding 'lm_eval/tasks/include/default/Croatian/include_base_44_croatian_stem.yaml' 2026-05-11T14:30:12,892 adding 'lm_eval/tasks/include/default/Croatian/utils.py' 2026-05-11T14:30:12,894 adding 'lm_eval/tasks/include/default/Dutch/_dutch_template_yaml' 2026-05-11T14:30:12,895 adding 'lm_eval/tasks/include/default/Dutch/_include_base_44_dutch.yaml' 2026-05-11T14:30:12,896 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_applied_science.yaml' 2026-05-11T14:30:12,897 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_arts_humanities.yaml' 2026-05-11T14:30:12,898 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_health_oriented_education.yaml' 2026-05-11T14:30:12,900 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_social_science.yaml' 2026-05-11T14:30:12,901 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_stem.yaml' 2026-05-11T14:30:12,902 adding 'lm_eval/tasks/include/default/Dutch/utils.py' 2026-05-11T14:30:12,904 adding 'lm_eval/tasks/include/default/Estonian/_estonian_template_yaml' 2026-05-11T14:30:12,905 adding 'lm_eval/tasks/include/default/Estonian/_include_base_44_estonian.yaml' 2026-05-11T14:30:12,906 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_applied_science.yaml' 2026-05-11T14:30:12,907 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_arts_humanities.yaml' 2026-05-11T14:30:12,908 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_health_oriented_education.yaml' 2026-05-11T14:30:12,909 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_social_science.yaml' 2026-05-11T14:30:12,911 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_stem.yaml' 2026-05-11T14:30:12,912 adding 'lm_eval/tasks/include/default/Estonian/utils.py' 2026-05-11T14:30:12,913 adding 'lm_eval/tasks/include/default/Finnish/_finnish_template_yaml' 2026-05-11T14:30:12,915 adding 'lm_eval/tasks/include/default/Finnish/_include_base_44_finnish.yaml' 2026-05-11T14:30:12,916 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_applied_science.yaml' 2026-05-11T14:30:12,917 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_arts_humanities.yaml' 2026-05-11T14:30:12,918 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_health_oriented_education.yaml' 2026-05-11T14:30:12,919 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_social_science.yaml' 2026-05-11T14:30:12,920 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_stem.yaml' 2026-05-11T14:30:12,921 adding 'lm_eval/tasks/include/default/Finnish/utils.py' 2026-05-11T14:30:12,923 adding 'lm_eval/tasks/include/default/French/_french_template_yaml' 2026-05-11T14:30:12,924 adding 'lm_eval/tasks/include/default/French/_include_base_44_french.yaml' 2026-05-11T14:30:12,925 adding 'lm_eval/tasks/include/default/French/include_base_44_french_arts_humanities.yaml' 2026-05-11T14:30:12,927 adding 'lm_eval/tasks/include/default/French/include_base_44_french_driving_license.yaml' 2026-05-11T14:30:12,928 adding 'lm_eval/tasks/include/default/French/include_base_44_french_health_oriented_education.yaml' 2026-05-11T14:30:12,929 adding 'lm_eval/tasks/include/default/French/include_base_44_french_social_science.yaml' 2026-05-11T14:30:12,930 adding 'lm_eval/tasks/include/default/French/include_base_44_french_stem.yaml' 2026-05-11T14:30:12,931 adding 'lm_eval/tasks/include/default/French/utils.py' 2026-05-11T14:30:12,933 adding 'lm_eval/tasks/include/default/Georgian/_georgian_template_yaml' 2026-05-11T14:30:12,934 adding 'lm_eval/tasks/include/default/Georgian/_include_base_44_georgian.yaml' 2026-05-11T14:30:12,935 adding 'lm_eval/tasks/include/default/Georgian/include_base_44_georgian_arts_humanities.yaml' 2026-05-11T14:30:12,937 adding 'lm_eval/tasks/include/default/Georgian/utils.py' 2026-05-11T14:30:12,938 adding 'lm_eval/tasks/include/default/German/_german_template_yaml' 2026-05-11T14:30:12,940 adding 'lm_eval/tasks/include/default/German/_include_base_44_german.yaml' 2026-05-11T14:30:12,941 adding 'lm_eval/tasks/include/default/German/include_base_44_german_driving_license.yaml' 2026-05-11T14:30:12,942 adding 'lm_eval/tasks/include/default/German/include_base_44_german_social_science.yaml' 2026-05-11T14:30:12,943 adding 'lm_eval/tasks/include/default/German/include_base_44_german_stem.yaml' 2026-05-11T14:30:12,944 adding 'lm_eval/tasks/include/default/German/utils.py' 2026-05-11T14:30:12,946 adding 'lm_eval/tasks/include/default/Greek/_greek_template_yaml' 2026-05-11T14:30:12,947 adding 'lm_eval/tasks/include/default/Greek/_include_base_44_greek.yaml' 2026-05-11T14:30:12,949 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_arts_humanities.yaml' 2026-05-11T14:30:12,950 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_business_commerce.yaml' 2026-05-11T14:30:12,951 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_health_oriented_education.yaml' 2026-05-11T14:30:12,952 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_medical_license.yaml' 2026-05-11T14:30:12,953 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_professional_certification.yaml' 2026-05-11T14:30:12,954 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_social_science.yaml' 2026-05-11T14:30:12,955 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_stem.yaml' 2026-05-11T14:30:12,957 adding 'lm_eval/tasks/include/default/Greek/utils.py' 2026-05-11T14:30:12,958 adding 'lm_eval/tasks/include/default/Hebrew/_hebrew_template_yaml' 2026-05-11T14:30:12,959 adding 'lm_eval/tasks/include/default/Hebrew/_include_base_44_hebrew.yaml' 2026-05-11T14:30:12,961 adding 'lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_arts_humanities.yaml' 2026-05-11T14:30:12,962 adding 'lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_driving_license.yaml' 2026-05-11T14:30:12,963 adding 'lm_eval/tasks/include/default/Hebrew/utils.py' 2026-05-11T14:30:12,965 adding 'lm_eval/tasks/include/default/Hindi/_hindi_template_yaml' 2026-05-11T14:30:12,966 adding 'lm_eval/tasks/include/default/Hindi/_include_base_44_hindi.yaml' 2026-05-11T14:30:12,967 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_applied_science.yaml' 2026-05-11T14:30:12,968 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_arts_humanities.yaml' 2026-05-11T14:30:12,969 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_driving_license.yaml' 2026-05-11T14:30:12,970 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_general_knowledge.yaml' 2026-05-11T14:30:12,972 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_health_oriented_education.yaml' 2026-05-11T14:30:12,973 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_professional_certification.yaml' 2026-05-11T14:30:12,974 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_social_science.yaml' 2026-05-11T14:30:12,975 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_stem.yaml' 2026-05-11T14:30:12,976 adding 'lm_eval/tasks/include/default/Hindi/utils.py' 2026-05-11T14:30:12,978 adding 'lm_eval/tasks/include/default/Hungarian/_hungarian_template_yaml' 2026-05-11T14:30:12,979 adding 'lm_eval/tasks/include/default/Hungarian/_include_base_44_hungarian.yaml' 2026-05-11T14:30:12,980 adding 'lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_applied_science.yaml' 2026-05-11T14:30:12,982 adding 'lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_social_science.yaml' 2026-05-11T14:30:12,983 adding 'lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_stem.yaml' 2026-05-11T14:30:12,984 adding 'lm_eval/tasks/include/default/Hungarian/utils.py' 2026-05-11T14:30:12,986 adding 'lm_eval/tasks/include/default/Indonesian/_include_base_44_indonesian.yaml' 2026-05-11T14:30:12,988 adding 'lm_eval/tasks/include/default/Indonesian/_indonesian_template_yaml' 2026-05-11T14:30:12,989 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_applied_science.yaml' 2026-05-11T14:30:12,990 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_arts_humanities.yaml' 2026-05-11T14:30:12,991 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_professional_certification.yaml' 2026-05-11T14:30:12,993 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_social_science.yaml' 2026-05-11T14:30:12,994 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_stem.yaml' 2026-05-11T14:30:12,995 adding 'lm_eval/tasks/include/default/Indonesian/utils.py' 2026-05-11T14:30:12,997 adding 'lm_eval/tasks/include/default/Italian/_include_base_44_italian.yaml' 2026-05-11T14:30:12,998 adding 'lm_eval/tasks/include/default/Italian/_italian_template_yaml' 2026-05-11T14:30:12,999 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_applied_science.yaml' 2026-05-11T14:30:13,000 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_arts_humanities.yaml' 2026-05-11T14:30:13,001 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_health_oriented_education.yaml' 2026-05-11T14:30:13,002 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_professional_certification.yaml' 2026-05-11T14:30:13,003 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_social_science.yaml' 2026-05-11T14:30:13,004 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_stem.yaml' 2026-05-11T14:30:13,006 adding 'lm_eval/tasks/include/default/Italian/utils.py' 2026-05-11T14:30:13,007 adding 'lm_eval/tasks/include/default/Japanese/_include_base_44_japanese.yaml' 2026-05-11T14:30:13,008 adding 'lm_eval/tasks/include/default/Japanese/_japanese_template_yaml' 2026-05-11T14:30:13,010 adding 'lm_eval/tasks/include/default/Japanese/include_base_44_japanese_driving_license.yaml' 2026-05-11T14:30:13,011 adding 'lm_eval/tasks/include/default/Japanese/include_base_44_japanese_medical_license.yaml' 2026-05-11T14:30:13,012 adding 'lm_eval/tasks/include/default/Japanese/include_base_44_japanese_professional_certification.yaml' 2026-05-11T14:30:13,013 adding 'lm_eval/tasks/include/default/Japanese/utils.py' 2026-05-11T14:30:13,015 adding 'lm_eval/tasks/include/default/Kazakh/_include_base_44_kazakh.yaml' 2026-05-11T14:30:13,016 adding 'lm_eval/tasks/include/default/Kazakh/_kazakh_template_yaml' 2026-05-11T14:30:13,017 adding 'lm_eval/tasks/include/default/Kazakh/include_base_44_kazakh_arts_humanities.yaml' 2026-05-11T14:30:13,019 adding 'lm_eval/tasks/include/default/Kazakh/utils.py' 2026-05-11T14:30:13,020 adding 'lm_eval/tasks/include/default/Korean/_include_base_44_korean.yaml' 2026-05-11T14:30:13,022 adding 'lm_eval/tasks/include/default/Korean/_korean_template_yaml' 2026-05-11T14:30:13,023 adding 'lm_eval/tasks/include/default/Korean/include_base_44_korean_professional_certification.yaml' 2026-05-11T14:30:13,024 adding 'lm_eval/tasks/include/default/Korean/include_base_44_korean_social_science.yaml' 2026-05-11T14:30:13,025 adding 'lm_eval/tasks/include/default/Korean/utils.py' 2026-05-11T14:30:13,027 adding 'lm_eval/tasks/include/default/Lithuanian/_include_base_44_lithuanian.yaml' 2026-05-11T14:30:13,028 adding 'lm_eval/tasks/include/default/Lithuanian/_lithuanian_template_yaml' 2026-05-11T14:30:13,029 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml' 2026-05-11T14:30:13,030 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_business_commerce.yaml' 2026-05-11T14:30:13,032 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_professional_certification.yaml' 2026-05-11T14:30:13,033 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_social_science.yaml' 2026-05-11T14:30:13,034 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_stem.yaml' 2026-05-11T14:30:13,035 adding 'lm_eval/tasks/include/default/Lithuanian/utils.py' 2026-05-11T14:30:13,037 adding 'lm_eval/tasks/include/default/Malay/_include_base_44_malay.yaml' 2026-05-11T14:30:13,038 adding 'lm_eval/tasks/include/default/Malay/_malay_template_yaml' 2026-05-11T14:30:13,039 adding 'lm_eval/tasks/include/default/Malay/include_base_44_malay_arts_humanities.yaml' 2026-05-11T14:30:13,040 adding 'lm_eval/tasks/include/default/Malay/include_base_44_malay_business_commerce.yaml' 2026-05-11T14:30:13,041 adding 'lm_eval/tasks/include/default/Malay/include_base_44_malay_social_science.yaml' 2026-05-11T14:30:13,043 adding 'lm_eval/tasks/include/default/Malay/utils.py' 2026-05-11T14:30:13,044 adding 'lm_eval/tasks/include/default/Malayalam/_include_base_44_malayalam.yaml' 2026-05-11T14:30:13,045 adding 'lm_eval/tasks/include/default/Malayalam/_malayalam_template_yaml' 2026-05-11T14:30:13,046 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_arts_humanities.yaml' 2026-05-11T14:30:13,047 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_general_knowledge.yaml' 2026-05-11T14:30:13,049 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_health_oriented_education.yaml' 2026-05-11T14:30:13,050 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_marine_license.yaml' 2026-05-11T14:30:13,051 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_social_science.yaml' 2026-05-11T14:30:13,052 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_stem.yaml' 2026-05-11T14:30:13,053 adding 'lm_eval/tasks/include/default/Malayalam/utils.py' 2026-05-11T14:30:13,055 adding 'lm_eval/tasks/include/default/Nepali/_include_base_44_nepali.yaml' 2026-05-11T14:30:13,056 adding 'lm_eval/tasks/include/default/Nepali/_nepali_template_yaml' 2026-05-11T14:30:13,057 adding 'lm_eval/tasks/include/default/Nepali/include_base_44_nepali_driving_license.yaml' 2026-05-11T14:30:13,058 adding 'lm_eval/tasks/include/default/Nepali/include_base_44_nepali_professional_certification.yaml' 2026-05-11T14:30:13,059 adding 'lm_eval/tasks/include/default/Nepali/utils.py' 2026-05-11T14:30:13,061 adding 'lm_eval/tasks/include/default/North Macedonian/_include_base_44_north macedonian.yaml' 2026-05-11T14:30:13,062 adding 'lm_eval/tasks/include/default/North Macedonian/_north macedonian_template_yaml' 2026-05-11T14:30:13,063 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml' 2026-05-11T14:30:13,065 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_business_commerce.yaml' 2026-05-11T14:30:13,066 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_social_science.yaml' 2026-05-11T14:30:13,067 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_stem.yaml' 2026-05-11T14:30:13,068 adding 'lm_eval/tasks/include/default/North Macedonian/utils.py' 2026-05-11T14:30:13,070 adding 'lm_eval/tasks/include/default/Persian/_include_base_44_persian.yaml' 2026-05-11T14:30:13,071 adding 'lm_eval/tasks/include/default/Persian/_persian_template_yaml' 2026-05-11T14:30:13,072 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_arts_humanities.yaml' 2026-05-11T14:30:13,073 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_driving_license.yaml' 2026-05-11T14:30:13,074 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_professional_certification.yaml' 2026-05-11T14:30:13,076 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_social_science.yaml' 2026-05-11T14:30:13,077 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_stem.yaml' 2026-05-11T14:30:13,078 adding 'lm_eval/tasks/include/default/Persian/utils.py' 2026-05-11T14:30:13,080 adding 'lm_eval/tasks/include/default/Polish/_include_base_44_polish.yaml' 2026-05-11T14:30:13,081 adding 'lm_eval/tasks/include/default/Polish/_polish_template_yaml' 2026-05-11T14:30:13,082 adding 'lm_eval/tasks/include/default/Polish/include_base_44_polish_professional_certification.yaml' 2026-05-11T14:30:13,083 adding 'lm_eval/tasks/include/default/Polish/include_base_44_polish_social_science.yaml' 2026-05-11T14:30:13,084 adding 'lm_eval/tasks/include/default/Polish/include_base_44_polish_stem.yaml' 2026-05-11T14:30:13,085 adding 'lm_eval/tasks/include/default/Polish/utils.py' 2026-05-11T14:30:13,087 adding 'lm_eval/tasks/include/default/Portuguese/_include_base_44_portuguese.yaml' 2026-05-11T14:30:13,088 adding 'lm_eval/tasks/include/default/Portuguese/_portuguese_template_yaml' 2026-05-11T14:30:13,089 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_applied_science.yaml' 2026-05-11T14:30:13,090 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_arts_humanities.yaml' 2026-05-11T14:30:13,091 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_business_commerce.yaml' 2026-05-11T14:30:13,092 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_health_oriented_education.yaml' 2026-05-11T14:30:13,094 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_social_science.yaml' 2026-05-11T14:30:13,095 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_stem.yaml' 2026-05-11T14:30:13,096 adding 'lm_eval/tasks/include/default/Portuguese/utils.py' 2026-05-11T14:30:13,097 adding 'lm_eval/tasks/include/default/Russian/_include_base_44_russian.yaml' 2026-05-11T14:30:13,099 adding 'lm_eval/tasks/include/default/Russian/_russian_template_yaml' 2026-05-11T14:30:13,100 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_applied_science.yaml' 2026-05-11T14:30:13,101 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_arts_humanities.yaml' 2026-05-11T14:30:13,102 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_business_commerce.yaml' 2026-05-11T14:30:13,103 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_driving_license.yaml' 2026-05-11T14:30:13,104 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_health_oriented_education.yaml' 2026-05-11T14:30:13,105 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_marine_license.yaml' 2026-05-11T14:30:13,106 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_social_science.yaml' 2026-05-11T14:30:13,108 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_stem.yaml' 2026-05-11T14:30:13,109 adding 'lm_eval/tasks/include/default/Russian/utils.py' 2026-05-11T14:30:13,111 adding 'lm_eval/tasks/include/default/Serbian/_include_base_44_serbian.yaml' 2026-05-11T14:30:13,112 adding 'lm_eval/tasks/include/default/Serbian/_serbian_template_yaml' 2026-05-11T14:30:13,113 adding 'lm_eval/tasks/include/default/Serbian/include_base_44_serbian_arts_humanities.yaml' 2026-05-11T14:30:13,114 adding 'lm_eval/tasks/include/default/Serbian/include_base_44_serbian_social_science.yaml' 2026-05-11T14:30:13,115 adding 'lm_eval/tasks/include/default/Serbian/include_base_44_serbian_stem.yaml' 2026-05-11T14:30:13,117 adding 'lm_eval/tasks/include/default/Serbian/utils.py' 2026-05-11T14:30:13,119 adding 'lm_eval/tasks/include/default/Spanish/_include_base_44_spanish.yaml' 2026-05-11T14:30:13,120 adding 'lm_eval/tasks/include/default/Spanish/_spanish_template_yaml' 2026-05-11T14:30:13,121 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_arts_humanities.yaml' 2026-05-11T14:30:13,122 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_health_oriented_education.yaml' 2026-05-11T14:30:13,123 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_social_science.yaml' 2026-05-11T14:30:13,125 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_stem.yaml' 2026-05-11T14:30:13,126 adding 'lm_eval/tasks/include/default/Spanish/utils.py' 2026-05-11T14:30:13,128 adding 'lm_eval/tasks/include/default/Tagalog/_include_base_44_tagalog.yaml' 2026-05-11T14:30:13,129 adding 'lm_eval/tasks/include/default/Tagalog/_tagalog_template_yaml' 2026-05-11T14:30:13,130 adding 'lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_arts_humanities.yaml' 2026-05-11T14:30:13,131 adding 'lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_driving_license.yaml' 2026-05-11T14:30:13,132 adding 'lm_eval/tasks/include/default/Tagalog/utils.py' 2026-05-11T14:30:13,134 adding 'lm_eval/tasks/include/default/Tamil/_include_base_44_tamil.yaml' 2026-05-11T14:30:13,135 adding 'lm_eval/tasks/include/default/Tamil/_tamil_template_yaml' 2026-05-11T14:30:13,136 adding 'lm_eval/tasks/include/default/Tamil/include_base_44_tamil_general_knowledge.yaml' 2026-05-11T14:30:13,137 adding 'lm_eval/tasks/include/default/Tamil/include_base_44_tamil_stem.yaml' 2026-05-11T14:30:13,138 adding 'lm_eval/tasks/include/default/Tamil/utils.py' 2026-05-11T14:30:13,140 adding 'lm_eval/tasks/include/default/Telugu/_include_base_44_telugu.yaml' 2026-05-11T14:30:13,141 adding 'lm_eval/tasks/include/default/Telugu/_telugu_template_yaml' 2026-05-11T14:30:13,142 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_applied_science.yaml' 2026-05-11T14:30:13,143 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_arts_humanities.yaml' 2026-05-11T14:30:13,144 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_social_science.yaml' 2026-05-11T14:30:13,145 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_stem.yaml' 2026-05-11T14:30:13,147 adding 'lm_eval/tasks/include/default/Telugu/utils.py' 2026-05-11T14:30:13,148 adding 'lm_eval/tasks/include/default/Turkish/_include_base_44_turkish.yaml' 2026-05-11T14:30:13,150 adding 'lm_eval/tasks/include/default/Turkish/_turkish_template_yaml' 2026-05-11T14:30:13,151 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_arts_humanities.yaml' 2026-05-11T14:30:13,152 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_business_commerce.yaml' 2026-05-11T14:30:13,153 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_social_science.yaml' 2026-05-11T14:30:13,154 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_stem.yaml' 2026-05-11T14:30:13,155 adding 'lm_eval/tasks/include/default/Turkish/utils.py' 2026-05-11T14:30:13,157 adding 'lm_eval/tasks/include/default/Ukrainian/_include_base_44_ukrainian.yaml' 2026-05-11T14:30:13,158 adding 'lm_eval/tasks/include/default/Ukrainian/_ukrainian_template_yaml' 2026-05-11T14:30:13,159 adding 'lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml' 2026-05-11T14:30:13,160 adding 'lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_social_science.yaml' 2026-05-11T14:30:13,162 adding 'lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_stem.yaml' 2026-05-11T14:30:13,163 adding 'lm_eval/tasks/include/default/Ukrainian/utils.py' 2026-05-11T14:30:13,164 adding 'lm_eval/tasks/include/default/Urdu/_include_base_44_urdu.yaml' 2026-05-11T14:30:13,165 adding 'lm_eval/tasks/include/default/Urdu/_urdu_template_yaml' 2026-05-11T14:30:13,167 adding 'lm_eval/tasks/include/default/Urdu/include_base_44_urdu_arts_humanities.yaml' 2026-05-11T14:30:13,168 adding 'lm_eval/tasks/include/default/Urdu/include_base_44_urdu_health_oriented_education.yaml' 2026-05-11T14:30:13,169 adding 'lm_eval/tasks/include/default/Urdu/include_base_44_urdu_stem.yaml' 2026-05-11T14:30:13,170 adding 'lm_eval/tasks/include/default/Urdu/utils.py' 2026-05-11T14:30:13,172 adding 'lm_eval/tasks/include/default/Uzbek/_include_base_44_uzbek.yaml' 2026-05-11T14:30:13,173 adding 'lm_eval/tasks/include/default/Uzbek/_uzbek_template_yaml' 2026-05-11T14:30:13,174 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_arts_humanities.yaml' 2026-05-11T14:30:13,175 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_medical_license.yaml' 2026-05-11T14:30:13,177 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_social_science.yaml' 2026-05-11T14:30:13,178 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_stem.yaml' 2026-05-11T14:30:13,179 adding 'lm_eval/tasks/include/default/Uzbek/utils.py' 2026-05-11T14:30:13,181 adding 'lm_eval/tasks/include/default/Vietnamese/_include_base_44_vietnamese.yaml' 2026-05-11T14:30:13,182 adding 'lm_eval/tasks/include/default/Vietnamese/_vietnamese_template_yaml' 2026-05-11T14:30:13,183 adding 'lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml' 2026-05-11T14:30:13,184 adding 'lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_social_science.yaml' 2026-05-11T14:30:13,185 adding 'lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_stem.yaml' 2026-05-11T14:30:13,187 adding 'lm_eval/tasks/include/default/Vietnamese/utils.py' 2026-05-11T14:30:13,189 adding 'lm_eval/tasks/include/few_shot_en/Albanian/_albanian_few_shot_en_template_yaml' 2026-05-11T14:30:13,190 adding 'lm_eval/tasks/include/few_shot_en/Albanian/_include_base_44_albanian.yaml' 2026-05-11T14:30:13,191 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_arts_humanities.yaml' 2026-05-11T14:30:13,192 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_business_commerce.yaml' 2026-05-11T14:30:13,193 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_health_oriented_education.yaml' 2026-05-11T14:30:13,194 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_social_science.yaml' 2026-05-11T14:30:13,196 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_stem.yaml' 2026-05-11T14:30:13,197 adding 'lm_eval/tasks/include/few_shot_en/Albanian/utils.py' 2026-05-11T14:30:13,199 adding 'lm_eval/tasks/include/few_shot_en/Arabic/_arabic_few_shot_en_template_yaml' 2026-05-11T14:30:13,200 adding 'lm_eval/tasks/include/few_shot_en/Arabic/_include_base_44_arabic.yaml' 2026-05-11T14:30:13,201 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_arts_humanities.yaml' 2026-05-11T14:30:13,202 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_business_commerce.yaml' 2026-05-11T14:30:13,203 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_driving_license.yaml' 2026-05-11T14:30:13,204 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_general_knowledge.yaml' 2026-05-11T14:30:13,205 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_social_science.yaml' 2026-05-11T14:30:13,207 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_stem.yaml' 2026-05-11T14:30:13,208 adding 'lm_eval/tasks/include/few_shot_en/Arabic/utils.py' 2026-05-11T14:30:13,210 adding 'lm_eval/tasks/include/few_shot_en/Armenian/_armenian_few_shot_en_template_yaml' 2026-05-11T14:30:13,211 adding 'lm_eval/tasks/include/few_shot_en/Armenian/_include_base_44_armenian.yaml' 2026-05-11T14:30:13,212 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_arts_humanities.yaml' 2026-05-11T14:30:13,213 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_driving_license.yaml' 2026-05-11T14:30:13,214 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_social_science.yaml' 2026-05-11T14:30:13,215 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_stem.yaml' 2026-05-11T14:30:13,217 adding 'lm_eval/tasks/include/few_shot_en/Armenian/utils.py' 2026-05-11T14:30:13,218 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/_azerbaijani_few_shot_en_template_yaml' 2026-05-11T14:30:13,220 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/_include_base_44_azerbaijani.yaml' 2026-05-11T14:30:13,221 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml' 2026-05-11T14:30:13,222 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml' 2026-05-11T14:30:13,223 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml' 2026-05-11T14:30:13,224 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml' 2026-05-11T14:30:13,225 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_social_science.yaml' 2026-05-11T14:30:13,227 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_stem.yaml' 2026-05-11T14:30:13,228 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/utils.py' 2026-05-11T14:30:13,230 adding 'lm_eval/tasks/include/few_shot_en/Basque/_basque_few_shot_en_template_yaml' 2026-05-11T14:30:13,231 adding 'lm_eval/tasks/include/few_shot_en/Basque/_include_base_44_basque.yaml' 2026-05-11T14:30:13,232 adding 'lm_eval/tasks/include/few_shot_en/Basque/include_base_44_basque_professional_certification.yaml' 2026-05-11T14:30:13,233 adding 'lm_eval/tasks/include/few_shot_en/Basque/utils.py' 2026-05-11T14:30:13,235 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/_belarusian_few_shot_en_template_yaml' 2026-05-11T14:30:13,236 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/_include_base_44_belarusian.yaml' 2026-05-11T14:30:13,237 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_arts_humanities.yaml' 2026-05-11T14:30:13,238 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_social_science.yaml' 2026-05-11T14:30:13,239 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_stem.yaml' 2026-05-11T14:30:13,240 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/utils.py' 2026-05-11T14:30:13,242 adding 'lm_eval/tasks/include/few_shot_en/Bengali/_bengali_few_shot_en_template_yaml' 2026-05-11T14:30:13,243 adding 'lm_eval/tasks/include/few_shot_en/Bengali/_include_base_44_bengali.yaml' 2026-05-11T14:30:13,244 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_arts_humanities.yaml' 2026-05-11T14:30:13,245 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_general_knowledge.yaml' 2026-05-11T14:30:13,247 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_professional_certification.yaml' 2026-05-11T14:30:13,248 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_stem.yaml' 2026-05-11T14:30:13,249 adding 'lm_eval/tasks/include/few_shot_en/Bengali/utils.py' 2026-05-11T14:30:13,251 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/_bulgarian_few_shot_en_template_yaml' 2026-05-11T14:30:13,252 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/_include_base_44_bulgarian.yaml' 2026-05-11T14:30:13,253 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml' 2026-05-11T14:30:13,254 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_social_science.yaml' 2026-05-11T14:30:13,255 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_stem.yaml' 2026-05-11T14:30:13,257 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/utils.py' 2026-05-11T14:30:13,258 adding 'lm_eval/tasks/include/few_shot_en/Chinese/_chinese_few_shot_en_template_yaml' 2026-05-11T14:30:13,260 adding 'lm_eval/tasks/include/few_shot_en/Chinese/_include_base_44_chinese.yaml' 2026-05-11T14:30:13,261 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_applied_science.yaml' 2026-05-11T14:30:13,262 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_arts_humanities.yaml' 2026-05-11T14:30:13,263 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_business_commerce.yaml' 2026-05-11T14:30:13,264 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_driving_license.yaml' 2026-05-11T14:30:13,266 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_health_oriented_education.yaml' 2026-05-11T14:30:13,267 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_professional_certification.yaml' 2026-05-11T14:30:13,268 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_social_science.yaml' 2026-05-11T14:30:13,269 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_stem.yaml' 2026-05-11T14:30:13,270 adding 'lm_eval/tasks/include/few_shot_en/Chinese/utils.py' 2026-05-11T14:30:13,272 adding 'lm_eval/tasks/include/few_shot_en/Croatian/_croatian_few_shot_en_template_yaml' 2026-05-11T14:30:13,273 adding 'lm_eval/tasks/include/few_shot_en/Croatian/_include_base_44_croatian.yaml' 2026-05-11T14:30:13,274 adding 'lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_arts_humanities.yaml' 2026-05-11T14:30:13,275 adding 'lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_social_science.yaml' 2026-05-11T14:30:13,276 adding 'lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_stem.yaml' 2026-05-11T14:30:13,277 adding 'lm_eval/tasks/include/few_shot_en/Croatian/utils.py' 2026-05-11T14:30:13,279 adding 'lm_eval/tasks/include/few_shot_en/Dutch/_dutch_few_shot_en_template_yaml' 2026-05-11T14:30:13,280 adding 'lm_eval/tasks/include/few_shot_en/Dutch/_include_base_44_dutch.yaml' 2026-05-11T14:30:13,281 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_applied_science.yaml' 2026-05-11T14:30:13,283 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_arts_humanities.yaml' 2026-05-11T14:30:13,284 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_health_oriented_education.yaml' 2026-05-11T14:30:13,285 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_social_science.yaml' 2026-05-11T14:30:13,286 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_stem.yaml' 2026-05-11T14:30:13,287 adding 'lm_eval/tasks/include/few_shot_en/Dutch/utils.py' 2026-05-11T14:30:13,289 adding 'lm_eval/tasks/include/few_shot_en/Estonian/_estonian_few_shot_en_template_yaml' 2026-05-11T14:30:13,291 adding 'lm_eval/tasks/include/few_shot_en/Estonian/_include_base_44_estonian.yaml' 2026-05-11T14:30:13,292 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_applied_science.yaml' 2026-05-11T14:30:13,293 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_arts_humanities.yaml' 2026-05-11T14:30:13,294 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_health_oriented_education.yaml' 2026-05-11T14:30:13,295 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_social_science.yaml' 2026-05-11T14:30:13,296 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_stem.yaml' 2026-05-11T14:30:13,298 adding 'lm_eval/tasks/include/few_shot_en/Estonian/utils.py' 2026-05-11T14:30:13,300 adding 'lm_eval/tasks/include/few_shot_en/Finnish/_finnish_few_shot_en_template_yaml' 2026-05-11T14:30:13,301 adding 'lm_eval/tasks/include/few_shot_en/Finnish/_include_base_44_finnish.yaml' 2026-05-11T14:30:13,302 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_applied_science.yaml' 2026-05-11T14:30:13,303 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_arts_humanities.yaml' 2026-05-11T14:30:13,304 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_health_oriented_education.yaml' 2026-05-11T14:30:13,306 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_social_science.yaml' 2026-05-11T14:30:13,307 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_stem.yaml' 2026-05-11T14:30:13,308 adding 'lm_eval/tasks/include/few_shot_en/Finnish/utils.py' 2026-05-11T14:30:13,310 adding 'lm_eval/tasks/include/few_shot_en/French/_french_few_shot_en_template_yaml' 2026-05-11T14:30:13,311 adding 'lm_eval/tasks/include/few_shot_en/French/_include_base_44_french.yaml' 2026-05-11T14:30:13,312 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_arts_humanities.yaml' 2026-05-11T14:30:13,313 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_driving_license.yaml' 2026-05-11T14:30:13,315 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_health_oriented_education.yaml' 2026-05-11T14:30:13,316 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_social_science.yaml' 2026-05-11T14:30:13,317 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_stem.yaml' 2026-05-11T14:30:13,318 adding 'lm_eval/tasks/include/few_shot_en/French/utils.py' 2026-05-11T14:30:13,320 adding 'lm_eval/tasks/include/few_shot_en/Georgian/_georgian_few_shot_en_template_yaml' 2026-05-11T14:30:13,321 adding 'lm_eval/tasks/include/few_shot_en/Georgian/_include_base_44_georgian.yaml' 2026-05-11T14:30:13,322 adding 'lm_eval/tasks/include/few_shot_en/Georgian/include_base_44_georgian_arts_humanities.yaml' 2026-05-11T14:30:13,323 adding 'lm_eval/tasks/include/few_shot_en/Georgian/utils.py' 2026-05-11T14:30:13,325 adding 'lm_eval/tasks/include/few_shot_en/German/_german_few_shot_en_template_yaml' 2026-05-11T14:30:13,326 adding 'lm_eval/tasks/include/few_shot_en/German/_include_base_44_german.yaml' 2026-05-11T14:30:13,327 adding 'lm_eval/tasks/include/few_shot_en/German/include_base_44_german_driving_license.yaml' 2026-05-11T14:30:13,328 adding 'lm_eval/tasks/include/few_shot_en/German/include_base_44_german_social_science.yaml' 2026-05-11T14:30:13,329 adding 'lm_eval/tasks/include/few_shot_en/German/include_base_44_german_stem.yaml' 2026-05-11T14:30:13,331 adding 'lm_eval/tasks/include/few_shot_en/German/utils.py' 2026-05-11T14:30:13,332 adding 'lm_eval/tasks/include/few_shot_en/Greek/_greek_few_shot_en_template_yaml' 2026-05-11T14:30:13,333 adding 'lm_eval/tasks/include/few_shot_en/Greek/_include_base_44_greek.yaml' 2026-05-11T14:30:13,334 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_arts_humanities.yaml' 2026-05-11T14:30:13,335 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_business_commerce.yaml' 2026-05-11T14:30:13,337 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_health_oriented_education.yaml' 2026-05-11T14:30:13,338 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_medical_license.yaml' 2026-05-11T14:30:13,339 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_professional_certification.yaml' 2026-05-11T14:30:13,340 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_social_science.yaml' 2026-05-11T14:30:13,342 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_stem.yaml' 2026-05-11T14:30:13,343 adding 'lm_eval/tasks/include/few_shot_en/Greek/utils.py' 2026-05-11T14:30:13,345 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/_hebrew_few_shot_en_template_yaml' 2026-05-11T14:30:13,346 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/_include_base_44_hebrew.yaml' 2026-05-11T14:30:13,347 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_arts_humanities.yaml' 2026-05-11T14:30:13,348 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_driving_license.yaml' 2026-05-11T14:30:13,350 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/utils.py' 2026-05-11T14:30:13,352 adding 'lm_eval/tasks/include/few_shot_en/Hindi/_hindi_few_shot_en_template_yaml' 2026-05-11T14:30:13,353 adding 'lm_eval/tasks/include/few_shot_en/Hindi/_include_base_44_hindi.yaml' 2026-05-11T14:30:13,354 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_applied_science.yaml' 2026-05-11T14:30:13,355 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_arts_humanities.yaml' 2026-05-11T14:30:13,357 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_driving_license.yaml' 2026-05-11T14:30:13,358 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_general_knowledge.yaml' 2026-05-11T14:30:13,359 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_health_oriented_education.yaml' 2026-05-11T14:30:13,360 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_professional_certification.yaml' 2026-05-11T14:30:13,361 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_social_science.yaml' 2026-05-11T14:30:13,362 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_stem.yaml' 2026-05-11T14:30:13,364 adding 'lm_eval/tasks/include/few_shot_en/Hindi/utils.py' 2026-05-11T14:30:13,365 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/_hungarian_few_shot_en_template_yaml' 2026-05-11T14:30:13,367 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/_include_base_44_hungarian.yaml' 2026-05-11T14:30:13,368 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_applied_science.yaml' 2026-05-11T14:30:13,369 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_social_science.yaml' 2026-05-11T14:30:13,370 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_stem.yaml' 2026-05-11T14:30:13,371 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/utils.py' 2026-05-11T14:30:13,373 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/_include_base_44_indonesian.yaml' 2026-05-11T14:30:13,374 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/_indonesian_few_shot_en_template_yaml' 2026-05-11T14:30:13,375 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_applied_science.yaml' 2026-05-11T14:30:13,376 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_arts_humanities.yaml' 2026-05-11T14:30:13,377 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_professional_certification.yaml' 2026-05-11T14:30:13,378 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_social_science.yaml' 2026-05-11T14:30:13,379 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_stem.yaml' 2026-05-11T14:30:13,381 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/utils.py' 2026-05-11T14:30:13,382 adding 'lm_eval/tasks/include/few_shot_en/Italian/_include_base_44_italian.yaml' 2026-05-11T14:30:13,384 adding 'lm_eval/tasks/include/few_shot_en/Italian/_italian_few_shot_en_template_yaml' 2026-05-11T14:30:13,385 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_applied_science.yaml' 2026-05-11T14:30:13,386 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_arts_humanities.yaml' 2026-05-11T14:30:13,388 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_health_oriented_education.yaml' 2026-05-11T14:30:13,389 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_professional_certification.yaml' 2026-05-11T14:30:13,391 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_social_science.yaml' 2026-05-11T14:30:13,392 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_stem.yaml' 2026-05-11T14:30:13,394 adding 'lm_eval/tasks/include/few_shot_en/Italian/utils.py' 2026-05-11T14:30:13,396 adding 'lm_eval/tasks/include/few_shot_en/Japanese/_include_base_44_japanese.yaml' 2026-05-11T14:30:13,397 adding 'lm_eval/tasks/include/few_shot_en/Japanese/_japanese_few_shot_en_template_yaml' 2026-05-11T14:30:13,398 adding 'lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_driving_license.yaml' 2026-05-11T14:30:13,400 adding 'lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_medical_license.yaml' 2026-05-11T14:30:13,401 adding 'lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_professional_certification.yaml' 2026-05-11T14:30:13,402 adding 'lm_eval/tasks/include/few_shot_en/Japanese/utils.py' 2026-05-11T14:30:13,404 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/_include_base_44_kazakh.yaml' 2026-05-11T14:30:13,405 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/_kazakh_few_shot_en_template_yaml' 2026-05-11T14:30:13,406 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/include_base_44_kazakh_arts_humanities.yaml' 2026-05-11T14:30:13,407 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/utils.py' 2026-05-11T14:30:13,409 adding 'lm_eval/tasks/include/few_shot_en/Korean/_include_base_44_korean.yaml' 2026-05-11T14:30:13,411 adding 'lm_eval/tasks/include/few_shot_en/Korean/_korean_few_shot_en_template_yaml' 2026-05-11T14:30:13,412 adding 'lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_professional_certification.yaml' 2026-05-11T14:30:13,413 adding 'lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_social_science.yaml' 2026-05-11T14:30:13,414 adding 'lm_eval/tasks/include/few_shot_en/Korean/utils.py' 2026-05-11T14:30:13,416 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/_include_base_44_lithuanian.yaml' 2026-05-11T14:30:13,417 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/_lithuanian_few_shot_en_template_yaml' 2026-05-11T14:30:13,419 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml' 2026-05-11T14:30:13,420 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_business_commerce.yaml' 2026-05-11T14:30:13,421 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_professional_certification.yaml' 2026-05-11T14:30:13,422 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_social_science.yaml' 2026-05-11T14:30:13,424 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_stem.yaml' 2026-05-11T14:30:13,425 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/utils.py' 2026-05-11T14:30:13,427 adding 'lm_eval/tasks/include/few_shot_en/Malay/_include_base_44_malay.yaml' 2026-05-11T14:30:13,428 adding 'lm_eval/tasks/include/few_shot_en/Malay/_malay_few_shot_en_template_yaml' 2026-05-11T14:30:13,429 adding 'lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_arts_humanities.yaml' 2026-05-11T14:30:13,430 adding 'lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_business_commerce.yaml' 2026-05-11T14:30:13,431 adding 'lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_social_science.yaml' 2026-05-11T14:30:13,432 adding 'lm_eval/tasks/include/few_shot_en/Malay/utils.py' 2026-05-11T14:30:13,434 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/_include_base_44_malayalam.yaml' 2026-05-11T14:30:13,435 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/_malayalam_few_shot_en_template_yaml' 2026-05-11T14:30:13,436 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_arts_humanities.yaml' 2026-05-11T14:30:13,438 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_general_knowledge.yaml' 2026-05-11T14:30:13,439 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_health_oriented_education.yaml' 2026-05-11T14:30:13,440 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_marine_license.yaml' 2026-05-11T14:30:13,441 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_social_science.yaml' 2026-05-11T14:30:13,442 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_stem.yaml' 2026-05-11T14:30:13,443 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/utils.py' 2026-05-11T14:30:13,445 adding 'lm_eval/tasks/include/few_shot_en/Nepali/_include_base_44_nepali.yaml' 2026-05-11T14:30:13,446 adding 'lm_eval/tasks/include/few_shot_en/Nepali/_nepali_few_shot_en_template_yaml' 2026-05-11T14:30:13,448 adding 'lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_driving_license.yaml' 2026-05-11T14:30:13,449 adding 'lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_professional_certification.yaml' 2026-05-11T14:30:13,450 adding 'lm_eval/tasks/include/few_shot_en/Nepali/utils.py' 2026-05-11T14:30:13,452 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/_include_base_44_north macedonian.yaml' 2026-05-11T14:30:13,453 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/_north macedonian_few_shot_en_template_yaml' 2026-05-11T14:30:13,454 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml' 2026-05-11T14:30:13,455 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_business_commerce.yaml' 2026-05-11T14:30:13,457 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_social_science.yaml' 2026-05-11T14:30:13,458 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_stem.yaml' 2026-05-11T14:30:13,459 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/utils.py' 2026-05-11T14:30:13,461 adding 'lm_eval/tasks/include/few_shot_en/Persian/_include_base_44_persian.yaml' 2026-05-11T14:30:13,462 adding 'lm_eval/tasks/include/few_shot_en/Persian/_persian_few_shot_en_template_yaml' 2026-05-11T14:30:13,463 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_arts_humanities.yaml' 2026-05-11T14:30:13,464 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_driving_license.yaml' 2026-05-11T14:30:13,466 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_professional_certification.yaml' 2026-05-11T14:30:13,467 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_social_science.yaml' 2026-05-11T14:30:13,468 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_stem.yaml' 2026-05-11T14:30:13,470 adding 'lm_eval/tasks/include/few_shot_en/Persian/utils.py' 2026-05-11T14:30:13,471 adding 'lm_eval/tasks/include/few_shot_en/Polish/_include_base_44_polish.yaml' 2026-05-11T14:30:13,472 adding 'lm_eval/tasks/include/few_shot_en/Polish/_polish_few_shot_en_template_yaml' 2026-05-11T14:30:13,473 adding 'lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_professional_certification.yaml' 2026-05-11T14:30:13,475 adding 'lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_social_science.yaml' 2026-05-11T14:30:13,476 adding 'lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_stem.yaml' 2026-05-11T14:30:13,477 adding 'lm_eval/tasks/include/few_shot_en/Polish/utils.py' 2026-05-11T14:30:13,479 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/_include_base_44_portuguese.yaml' 2026-05-11T14:30:13,480 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/_portuguese_few_shot_en_template_yaml' 2026-05-11T14:30:13,481 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_applied_science.yaml' 2026-05-11T14:30:13,482 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_arts_humanities.yaml' 2026-05-11T14:30:13,484 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_business_commerce.yaml' 2026-05-11T14:30:13,485 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_health_oriented_education.yaml' 2026-05-11T14:30:13,486 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_social_science.yaml' 2026-05-11T14:30:13,487 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_stem.yaml' 2026-05-11T14:30:13,488 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/utils.py' 2026-05-11T14:30:13,490 adding 'lm_eval/tasks/include/few_shot_en/Russian/_include_base_44_russian.yaml' 2026-05-11T14:30:13,492 adding 'lm_eval/tasks/include/few_shot_en/Russian/_russian_few_shot_en_template_yaml' 2026-05-11T14:30:13,493 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_applied_science.yaml' 2026-05-11T14:30:13,494 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_arts_humanities.yaml' 2026-05-11T14:30:13,495 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_business_commerce.yaml' 2026-05-11T14:30:13,496 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_driving_license.yaml' 2026-05-11T14:30:13,498 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_health_oriented_education.yaml' 2026-05-11T14:30:13,499 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_marine_license.yaml' 2026-05-11T14:30:13,500 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_social_science.yaml' 2026-05-11T14:30:13,502 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_stem.yaml' 2026-05-11T14:30:13,503 adding 'lm_eval/tasks/include/few_shot_en/Russian/utils.py' 2026-05-11T14:30:13,504 adding 'lm_eval/tasks/include/few_shot_en/Serbian/_include_base_44_serbian.yaml' 2026-05-11T14:30:13,506 adding 'lm_eval/tasks/include/few_shot_en/Serbian/_serbian_few_shot_en_template_yaml' 2026-05-11T14:30:13,513 adding 'lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_arts_humanities.yaml' 2026-05-11T14:30:13,515 adding 'lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_social_science.yaml' 2026-05-11T14:30:13,516 adding 'lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_stem.yaml' 2026-05-11T14:30:13,517 adding 'lm_eval/tasks/include/few_shot_en/Serbian/utils.py' 2026-05-11T14:30:13,519 adding 'lm_eval/tasks/include/few_shot_en/Spanish/_include_base_44_spanish.yaml' 2026-05-11T14:30:13,520 adding 'lm_eval/tasks/include/few_shot_en/Spanish/_spanish_few_shot_en_template_yaml' 2026-05-11T14:30:13,521 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_arts_humanities.yaml' 2026-05-11T14:30:13,523 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_health_oriented_education.yaml' 2026-05-11T14:30:13,524 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_social_science.yaml' 2026-05-11T14:30:13,525 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_stem.yaml' 2026-05-11T14:30:13,527 adding 'lm_eval/tasks/include/few_shot_en/Spanish/utils.py' 2026-05-11T14:30:13,530 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/_include_base_44_tagalog.yaml' 2026-05-11T14:30:13,532 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/_tagalog_few_shot_en_template_yaml' 2026-05-11T14:30:13,534 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_arts_humanities.yaml' 2026-05-11T14:30:13,535 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_driving_license.yaml' 2026-05-11T14:30:13,537 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/utils.py' 2026-05-11T14:30:13,538 adding 'lm_eval/tasks/include/few_shot_en/Tamil/_include_base_44_tamil.yaml' 2026-05-11T14:30:13,540 adding 'lm_eval/tasks/include/few_shot_en/Tamil/_tamil_few_shot_en_template_yaml' 2026-05-11T14:30:13,541 adding 'lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_general_knowledge.yaml' 2026-05-11T14:30:13,542 adding 'lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_stem.yaml' 2026-05-11T14:30:13,543 adding 'lm_eval/tasks/include/few_shot_en/Tamil/utils.py' 2026-05-11T14:30:13,545 adding 'lm_eval/tasks/include/few_shot_en/Telugu/_include_base_44_telugu.yaml' 2026-05-11T14:30:13,546 adding 'lm_eval/tasks/include/few_shot_en/Telugu/_telugu_few_shot_en_template_yaml' 2026-05-11T14:30:13,547 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_applied_science.yaml' 2026-05-11T14:30:13,549 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_arts_humanities.yaml' 2026-05-11T14:30:13,550 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_social_science.yaml' 2026-05-11T14:30:13,551 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_stem.yaml' 2026-05-11T14:30:13,552 adding 'lm_eval/tasks/include/few_shot_en/Telugu/utils.py' 2026-05-11T14:30:13,554 adding 'lm_eval/tasks/include/few_shot_en/Turkish/_include_base_44_turkish.yaml' 2026-05-11T14:30:13,555 adding 'lm_eval/tasks/include/few_shot_en/Turkish/_turkish_few_shot_en_template_yaml' 2026-05-11T14:30:13,556 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_arts_humanities.yaml' 2026-05-11T14:30:13,557 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_business_commerce.yaml' 2026-05-11T14:30:13,559 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_social_science.yaml' 2026-05-11T14:30:13,560 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_stem.yaml' 2026-05-11T14:30:13,561 adding 'lm_eval/tasks/include/few_shot_en/Turkish/utils.py' 2026-05-11T14:30:13,562 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/_include_base_44_ukrainian.yaml' 2026-05-11T14:30:13,564 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/_ukrainian_few_shot_en_template_yaml' 2026-05-11T14:30:13,565 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml' 2026-05-11T14:30:13,566 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_social_science.yaml' 2026-05-11T14:30:13,567 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_stem.yaml' 2026-05-11T14:30:13,568 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/utils.py' 2026-05-11T14:30:13,569 adding 'lm_eval/tasks/include/few_shot_en/Urdu/_include_base_44_urdu.yaml' 2026-05-11T14:30:13,571 adding 'lm_eval/tasks/include/few_shot_en/Urdu/_urdu_few_shot_en_template_yaml' 2026-05-11T14:30:13,572 adding 'lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_arts_humanities.yaml' 2026-05-11T14:30:13,573 adding 'lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_health_oriented_education.yaml' 2026-05-11T14:30:13,574 adding 'lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_stem.yaml' 2026-05-11T14:30:13,575 adding 'lm_eval/tasks/include/few_shot_en/Urdu/utils.py' 2026-05-11T14:30:13,577 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/_include_base_44_uzbek.yaml' 2026-05-11T14:30:13,578 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/_uzbek_few_shot_en_template_yaml' 2026-05-11T14:30:13,579 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_arts_humanities.yaml' 2026-05-11T14:30:13,580 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_medical_license.yaml' 2026-05-11T14:30:13,581 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_social_science.yaml' 2026-05-11T14:30:13,582 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_stem.yaml' 2026-05-11T14:30:13,584 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/utils.py' 2026-05-11T14:30:13,585 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/_include_base_44_vietnamese.yaml' 2026-05-11T14:30:13,587 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/_vietnamese_few_shot_en_template_yaml' 2026-05-11T14:30:13,588 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml' 2026-05-11T14:30:13,589 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_social_science.yaml' 2026-05-11T14:30:13,590 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_stem.yaml' 2026-05-11T14:30:13,592 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/utils.py' 2026-05-11T14:30:13,594 adding 'lm_eval/tasks/include/few_shot_og/Albanian/_albanian_few_shot_og_template_yaml' 2026-05-11T14:30:13,597 adding 'lm_eval/tasks/include/few_shot_og/Albanian/_include_base_44_albanian.yaml' 2026-05-11T14:30:13,599 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_arts_humanities.yaml' 2026-05-11T14:30:13,600 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_business_commerce.yaml' 2026-05-11T14:30:13,601 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_health_oriented_education.yaml' 2026-05-11T14:30:13,602 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_social_science.yaml' 2026-05-11T14:30:13,603 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_stem.yaml' 2026-05-11T14:30:13,605 adding 'lm_eval/tasks/include/few_shot_og/Albanian/utils.py' 2026-05-11T14:30:13,606 adding 'lm_eval/tasks/include/few_shot_og/Arabic/_arabic_few_shot_og_template_yaml' 2026-05-11T14:30:13,608 adding 'lm_eval/tasks/include/few_shot_og/Arabic/_include_base_44_arabic.yaml' 2026-05-11T14:30:13,609 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_arts_humanities.yaml' 2026-05-11T14:30:13,610 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_business_commerce.yaml' 2026-05-11T14:30:13,611 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_driving_license.yaml' 2026-05-11T14:30:13,612 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_general_knowledge.yaml' 2026-05-11T14:30:13,613 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_social_science.yaml' 2026-05-11T14:30:13,614 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_stem.yaml' 2026-05-11T14:30:13,615 adding 'lm_eval/tasks/include/few_shot_og/Arabic/utils.py' 2026-05-11T14:30:13,617 adding 'lm_eval/tasks/include/few_shot_og/Armenian/_armenian_few_shot_og_template_yaml' 2026-05-11T14:30:13,618 adding 'lm_eval/tasks/include/few_shot_og/Armenian/_include_base_44_armenian.yaml' 2026-05-11T14:30:13,619 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_arts_humanities.yaml' 2026-05-11T14:30:13,620 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_driving_license.yaml' 2026-05-11T14:30:13,622 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_social_science.yaml' 2026-05-11T14:30:13,623 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_stem.yaml' 2026-05-11T14:30:13,624 adding 'lm_eval/tasks/include/few_shot_og/Armenian/utils.py' 2026-05-11T14:30:13,626 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/_azerbaijani_few_shot_og_template_yaml' 2026-05-11T14:30:13,627 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/_include_base_44_azerbaijani.yaml' 2026-05-11T14:30:13,628 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml' 2026-05-11T14:30:13,630 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml' 2026-05-11T14:30:13,631 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml' 2026-05-11T14:30:13,632 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml' 2026-05-11T14:30:13,634 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_social_science.yaml' 2026-05-11T14:30:13,635 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_stem.yaml' 2026-05-11T14:30:13,636 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/utils.py' 2026-05-11T14:30:13,638 adding 'lm_eval/tasks/include/few_shot_og/Basque/_basque_few_shot_og_template_yaml' 2026-05-11T14:30:13,639 adding 'lm_eval/tasks/include/few_shot_og/Basque/_include_base_44_basque.yaml' 2026-05-11T14:30:13,640 adding 'lm_eval/tasks/include/few_shot_og/Basque/include_base_44_basque_professional_certification.yaml' 2026-05-11T14:30:13,641 adding 'lm_eval/tasks/include/few_shot_og/Basque/utils.py' 2026-05-11T14:30:13,643 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/_belarusian_few_shot_og_template_yaml' 2026-05-11T14:30:13,644 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/_include_base_44_belarusian.yaml' 2026-05-11T14:30:13,645 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_arts_humanities.yaml' 2026-05-11T14:30:13,647 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_social_science.yaml' 2026-05-11T14:30:13,648 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_stem.yaml' 2026-05-11T14:30:13,649 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/utils.py' 2026-05-11T14:30:13,651 adding 'lm_eval/tasks/include/few_shot_og/Bengali/_bengali_few_shot_og_template_yaml' 2026-05-11T14:30:13,652 adding 'lm_eval/tasks/include/few_shot_og/Bengali/_include_base_44_bengali.yaml' 2026-05-11T14:30:13,654 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_arts_humanities.yaml' 2026-05-11T14:30:13,655 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_general_knowledge.yaml' 2026-05-11T14:30:13,656 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_professional_certification.yaml' 2026-05-11T14:30:13,657 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_stem.yaml' 2026-05-11T14:30:13,658 adding 'lm_eval/tasks/include/few_shot_og/Bengali/utils.py' 2026-05-11T14:30:13,660 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/_bulgarian_few_shot_og_template_yaml' 2026-05-11T14:30:13,661 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/_include_base_44_bulgarian.yaml' 2026-05-11T14:30:13,662 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml' 2026-05-11T14:30:13,664 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_social_science.yaml' 2026-05-11T14:30:13,665 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_stem.yaml' 2026-05-11T14:30:13,666 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/utils.py' 2026-05-11T14:30:13,668 adding 'lm_eval/tasks/include/few_shot_og/Chinese/_chinese_few_shot_og_template_yaml' 2026-05-11T14:30:13,669 adding 'lm_eval/tasks/include/few_shot_og/Chinese/_include_base_44_chinese.yaml' 2026-05-11T14:30:13,670 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_applied_science.yaml' 2026-05-11T14:30:13,671 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_arts_humanities.yaml' 2026-05-11T14:30:13,672 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_business_commerce.yaml' 2026-05-11T14:30:13,674 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_driving_license.yaml' 2026-05-11T14:30:13,675 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_health_oriented_education.yaml' 2026-05-11T14:30:13,676 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_professional_certification.yaml' 2026-05-11T14:30:13,677 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_social_science.yaml' 2026-05-11T14:30:13,678 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_stem.yaml' 2026-05-11T14:30:13,680 adding 'lm_eval/tasks/include/few_shot_og/Chinese/utils.py' 2026-05-11T14:30:13,682 adding 'lm_eval/tasks/include/few_shot_og/Croatian/_croatian_few_shot_og_template_yaml' 2026-05-11T14:30:13,683 adding 'lm_eval/tasks/include/few_shot_og/Croatian/_include_base_44_croatian.yaml' 2026-05-11T14:30:13,684 adding 'lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_arts_humanities.yaml' 2026-05-11T14:30:13,685 adding 'lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_social_science.yaml' 2026-05-11T14:30:13,687 adding 'lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_stem.yaml' 2026-05-11T14:30:13,688 adding 'lm_eval/tasks/include/few_shot_og/Croatian/utils.py' 2026-05-11T14:30:13,690 adding 'lm_eval/tasks/include/few_shot_og/Dutch/_dutch_few_shot_og_template_yaml' 2026-05-11T14:30:13,691 adding 'lm_eval/tasks/include/few_shot_og/Dutch/_include_base_44_dutch.yaml' 2026-05-11T14:30:13,692 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_applied_science.yaml' 2026-05-11T14:30:13,693 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_arts_humanities.yaml' 2026-05-11T14:30:13,695 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_health_oriented_education.yaml' 2026-05-11T14:30:13,696 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_social_science.yaml' 2026-05-11T14:30:13,697 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_stem.yaml' 2026-05-11T14:30:13,698 adding 'lm_eval/tasks/include/few_shot_og/Dutch/utils.py' 2026-05-11T14:30:13,700 adding 'lm_eval/tasks/include/few_shot_og/Estonian/_estonian_few_shot_og_template_yaml' 2026-05-11T14:30:13,701 adding 'lm_eval/tasks/include/few_shot_og/Estonian/_include_base_44_estonian.yaml' 2026-05-11T14:30:13,702 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_applied_science.yaml' 2026-05-11T14:30:13,703 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_arts_humanities.yaml' 2026-05-11T14:30:13,705 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_health_oriented_education.yaml' 2026-05-11T14:30:13,706 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_social_science.yaml' 2026-05-11T14:30:13,707 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_stem.yaml' 2026-05-11T14:30:13,708 adding 'lm_eval/tasks/include/few_shot_og/Estonian/utils.py' 2026-05-11T14:30:13,710 adding 'lm_eval/tasks/include/few_shot_og/Finnish/_finnish_few_shot_og_template_yaml' 2026-05-11T14:30:13,711 adding 'lm_eval/tasks/include/few_shot_og/Finnish/_include_base_44_finnish.yaml' 2026-05-11T14:30:13,712 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_applied_science.yaml' 2026-05-11T14:30:13,713 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_arts_humanities.yaml' 2026-05-11T14:30:13,715 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_health_oriented_education.yaml' 2026-05-11T14:30:13,716 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_social_science.yaml' 2026-05-11T14:30:13,717 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_stem.yaml' 2026-05-11T14:30:13,718 adding 'lm_eval/tasks/include/few_shot_og/Finnish/utils.py' 2026-05-11T14:30:13,720 adding 'lm_eval/tasks/include/few_shot_og/French/_french_few_shot_og_template_yaml' 2026-05-11T14:30:13,721 adding 'lm_eval/tasks/include/few_shot_og/French/_include_base_44_french.yaml' 2026-05-11T14:30:13,722 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_arts_humanities.yaml' 2026-05-11T14:30:13,723 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_driving_license.yaml' 2026-05-11T14:30:13,724 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_health_oriented_education.yaml' 2026-05-11T14:30:13,726 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_social_science.yaml' 2026-05-11T14:30:13,727 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_stem.yaml' 2026-05-11T14:30:13,728 adding 'lm_eval/tasks/include/few_shot_og/French/utils.py' 2026-05-11T14:30:13,730 adding 'lm_eval/tasks/include/few_shot_og/Georgian/_georgian_few_shot_og_template_yaml' 2026-05-11T14:30:13,731 adding 'lm_eval/tasks/include/few_shot_og/Georgian/_include_base_44_georgian.yaml' 2026-05-11T14:30:13,732 adding 'lm_eval/tasks/include/few_shot_og/Georgian/include_base_44_georgian_arts_humanities.yaml' 2026-05-11T14:30:13,733 adding 'lm_eval/tasks/include/few_shot_og/Georgian/utils.py' 2026-05-11T14:30:13,735 adding 'lm_eval/tasks/include/few_shot_og/German/_german_few_shot_og_template_yaml' 2026-05-11T14:30:13,737 adding 'lm_eval/tasks/include/few_shot_og/German/_include_base_44_german.yaml' 2026-05-11T14:30:13,738 adding 'lm_eval/tasks/include/few_shot_og/German/include_base_44_german_driving_license.yaml' 2026-05-11T14:30:13,739 adding 'lm_eval/tasks/include/few_shot_og/German/include_base_44_german_social_science.yaml' 2026-05-11T14:30:13,740 adding 'lm_eval/tasks/include/few_shot_og/German/include_base_44_german_stem.yaml' 2026-05-11T14:30:13,742 adding 'lm_eval/tasks/include/few_shot_og/German/utils.py' 2026-05-11T14:30:13,743 adding 'lm_eval/tasks/include/few_shot_og/Greek/_greek_few_shot_og_template_yaml' 2026-05-11T14:30:13,745 adding 'lm_eval/tasks/include/few_shot_og/Greek/_include_base_44_greek.yaml' 2026-05-11T14:30:13,746 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_arts_humanities.yaml' 2026-05-11T14:30:13,747 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_business_commerce.yaml' 2026-05-11T14:30:13,748 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_health_oriented_education.yaml' 2026-05-11T14:30:13,749 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_medical_license.yaml' 2026-05-11T14:30:13,750 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_professional_certification.yaml' 2026-05-11T14:30:13,752 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_social_science.yaml' 2026-05-11T14:30:13,753 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_stem.yaml' 2026-05-11T14:30:13,754 adding 'lm_eval/tasks/include/few_shot_og/Greek/utils.py' 2026-05-11T14:30:13,756 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/_hebrew_few_shot_og_template_yaml' 2026-05-11T14:30:13,757 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/_include_base_44_hebrew.yaml' 2026-05-11T14:30:13,758 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_arts_humanities.yaml' 2026-05-11T14:30:13,759 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_driving_license.yaml' 2026-05-11T14:30:13,761 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/utils.py' 2026-05-11T14:30:13,763 adding 'lm_eval/tasks/include/few_shot_og/Hindi/_hindi_few_shot_og_template_yaml' 2026-05-11T14:30:13,764 adding 'lm_eval/tasks/include/few_shot_og/Hindi/_include_base_44_hindi.yaml' 2026-05-11T14:30:13,765 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_applied_science.yaml' 2026-05-11T14:30:13,766 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_arts_humanities.yaml' 2026-05-11T14:30:13,767 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_driving_license.yaml' 2026-05-11T14:30:13,769 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_general_knowledge.yaml' 2026-05-11T14:30:13,770 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_health_oriented_education.yaml' 2026-05-11T14:30:13,771 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_professional_certification.yaml' 2026-05-11T14:30:13,772 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_social_science.yaml' 2026-05-11T14:30:13,773 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_stem.yaml' 2026-05-11T14:30:13,774 adding 'lm_eval/tasks/include/few_shot_og/Hindi/utils.py' 2026-05-11T14:30:13,776 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/_hungarian_few_shot_og_template_yaml' 2026-05-11T14:30:13,778 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/_include_base_44_hungarian.yaml' 2026-05-11T14:30:13,779 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_applied_science.yaml' 2026-05-11T14:30:13,780 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_social_science.yaml' 2026-05-11T14:30:13,781 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_stem.yaml' 2026-05-11T14:30:13,783 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/utils.py' 2026-05-11T14:30:13,784 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/_include_base_44_indonesian.yaml' 2026-05-11T14:30:13,786 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/_indonesian_few_shot_og_template_yaml' 2026-05-11T14:30:13,787 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_applied_science.yaml' 2026-05-11T14:30:13,788 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_arts_humanities.yaml' 2026-05-11T14:30:13,789 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_professional_certification.yaml' 2026-05-11T14:30:13,790 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_social_science.yaml' 2026-05-11T14:30:13,792 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_stem.yaml' 2026-05-11T14:30:13,793 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/utils.py' 2026-05-11T14:30:13,795 adding 'lm_eval/tasks/include/few_shot_og/Italian/_include_base_44_italian.yaml' 2026-05-11T14:30:13,796 adding 'lm_eval/tasks/include/few_shot_og/Italian/_italian_few_shot_og_template_yaml' 2026-05-11T14:30:13,797 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_applied_science.yaml' 2026-05-11T14:30:13,798 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_arts_humanities.yaml' 2026-05-11T14:30:13,799 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_health_oriented_education.yaml' 2026-05-11T14:30:13,800 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_professional_certification.yaml' 2026-05-11T14:30:13,801 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_social_science.yaml' 2026-05-11T14:30:13,803 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_stem.yaml' 2026-05-11T14:30:13,804 adding 'lm_eval/tasks/include/few_shot_og/Italian/utils.py' 2026-05-11T14:30:13,806 adding 'lm_eval/tasks/include/few_shot_og/Japanese/_include_base_44_japanese.yaml' 2026-05-11T14:30:13,807 adding 'lm_eval/tasks/include/few_shot_og/Japanese/_japanese_few_shot_og_template_yaml' 2026-05-11T14:30:13,808 adding 'lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_driving_license.yaml' 2026-05-11T14:30:13,809 adding 'lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_medical_license.yaml' 2026-05-11T14:30:13,810 adding 'lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_professional_certification.yaml' 2026-05-11T14:30:13,812 adding 'lm_eval/tasks/include/few_shot_og/Japanese/utils.py' 2026-05-11T14:30:13,813 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/_include_base_44_kazakh.yaml' 2026-05-11T14:30:13,815 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/_kazakh_few_shot_og_template_yaml' 2026-05-11T14:30:13,816 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/include_base_44_kazakh_arts_humanities.yaml' 2026-05-11T14:30:13,817 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/utils.py' 2026-05-11T14:30:13,819 adding 'lm_eval/tasks/include/few_shot_og/Korean/_include_base_44_korean.yaml' 2026-05-11T14:30:13,820 adding 'lm_eval/tasks/include/few_shot_og/Korean/_korean_few_shot_og_template_yaml' 2026-05-11T14:30:13,821 adding 'lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_professional_certification.yaml' 2026-05-11T14:30:13,822 adding 'lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_social_science.yaml' 2026-05-11T14:30:13,824 adding 'lm_eval/tasks/include/few_shot_og/Korean/utils.py' 2026-05-11T14:30:13,826 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/_include_base_44_lithuanian.yaml' 2026-05-11T14:30:13,827 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/_lithuanian_few_shot_og_template_yaml' 2026-05-11T14:30:13,828 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml' 2026-05-11T14:30:13,829 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_business_commerce.yaml' 2026-05-11T14:30:13,831 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_professional_certification.yaml' 2026-05-11T14:30:13,832 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_social_science.yaml' 2026-05-11T14:30:13,833 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_stem.yaml' 2026-05-11T14:30:13,834 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/utils.py' 2026-05-11T14:30:13,836 adding 'lm_eval/tasks/include/few_shot_og/Malay/_include_base_44_malay.yaml' 2026-05-11T14:30:13,837 adding 'lm_eval/tasks/include/few_shot_og/Malay/_malay_few_shot_og_template_yaml' 2026-05-11T14:30:13,839 adding 'lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_arts_humanities.yaml' 2026-05-11T14:30:13,840 adding 'lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_business_commerce.yaml' 2026-05-11T14:30:13,841 adding 'lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_social_science.yaml' 2026-05-11T14:30:13,842 adding 'lm_eval/tasks/include/few_shot_og/Malay/utils.py' 2026-05-11T14:30:13,844 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/_include_base_44_malayalam.yaml' 2026-05-11T14:30:13,845 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/_malayalam_few_shot_og_template_yaml' 2026-05-11T14:30:13,846 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_arts_humanities.yaml' 2026-05-11T14:30:13,847 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_general_knowledge.yaml' 2026-05-11T14:30:13,848 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_health_oriented_education.yaml' 2026-05-11T14:30:13,850 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_marine_license.yaml' 2026-05-11T14:30:13,851 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_social_science.yaml' 2026-05-11T14:30:13,852 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_stem.yaml' 2026-05-11T14:30:13,853 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/utils.py' 2026-05-11T14:30:13,855 adding 'lm_eval/tasks/include/few_shot_og/Nepali/_include_base_44_nepali.yaml' 2026-05-11T14:30:13,856 adding 'lm_eval/tasks/include/few_shot_og/Nepali/_nepali_few_shot_og_template_yaml' 2026-05-11T14:30:13,858 adding 'lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_driving_license.yaml' 2026-05-11T14:30:13,859 adding 'lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_professional_certification.yaml' 2026-05-11T14:30:13,860 adding 'lm_eval/tasks/include/few_shot_og/Nepali/utils.py' 2026-05-11T14:30:13,862 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/_include_base_44_north macedonian.yaml' 2026-05-11T14:30:13,863 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/_north macedonian_few_shot_og_template_yaml' 2026-05-11T14:30:13,864 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml' 2026-05-11T14:30:13,865 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_business_commerce.yaml' 2026-05-11T14:30:13,867 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_social_science.yaml' 2026-05-11T14:30:13,868 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_stem.yaml' 2026-05-11T14:30:13,869 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/utils.py' 2026-05-11T14:30:13,871 adding 'lm_eval/tasks/include/few_shot_og/Persian/_include_base_44_persian.yaml' 2026-05-11T14:30:13,872 adding 'lm_eval/tasks/include/few_shot_og/Persian/_persian_few_shot_og_template_yaml' 2026-05-11T14:30:13,873 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_arts_humanities.yaml' 2026-05-11T14:30:13,875 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_driving_license.yaml' 2026-05-11T14:30:13,876 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_professional_certification.yaml' 2026-05-11T14:30:13,877 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_social_science.yaml' 2026-05-11T14:30:13,878 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_stem.yaml' 2026-05-11T14:30:13,880 adding 'lm_eval/tasks/include/few_shot_og/Persian/utils.py' 2026-05-11T14:30:13,881 adding 'lm_eval/tasks/include/few_shot_og/Polish/_include_base_44_polish.yaml' 2026-05-11T14:30:13,883 adding 'lm_eval/tasks/include/few_shot_og/Polish/_polish_few_shot_og_template_yaml' 2026-05-11T14:30:13,884 adding 'lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_professional_certification.yaml' 2026-05-11T14:30:13,885 adding 'lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_social_science.yaml' 2026-05-11T14:30:13,886 adding 'lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_stem.yaml' 2026-05-11T14:30:13,887 adding 'lm_eval/tasks/include/few_shot_og/Polish/utils.py' 2026-05-11T14:30:13,889 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/_include_base_44_portuguese.yaml' 2026-05-11T14:30:13,890 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/_portuguese_few_shot_og_template_yaml' 2026-05-11T14:30:13,891 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_applied_science.yaml' 2026-05-11T14:30:13,892 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_arts_humanities.yaml' 2026-05-11T14:30:13,893 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_business_commerce.yaml' 2026-05-11T14:30:13,895 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_health_oriented_education.yaml' 2026-05-11T14:30:13,896 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_social_science.yaml' 2026-05-11T14:30:13,897 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_stem.yaml' 2026-05-11T14:30:13,898 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/utils.py' 2026-05-11T14:30:13,900 adding 'lm_eval/tasks/include/few_shot_og/Russian/_include_base_44_russian.yaml' 2026-05-11T14:30:13,901 adding 'lm_eval/tasks/include/few_shot_og/Russian/_russian_few_shot_og_template_yaml' 2026-05-11T14:30:13,903 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_applied_science.yaml' 2026-05-11T14:30:13,904 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_arts_humanities.yaml' 2026-05-11T14:30:13,905 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_business_commerce.yaml' 2026-05-11T14:30:13,906 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_driving_license.yaml' 2026-05-11T14:30:13,907 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_health_oriented_education.yaml' 2026-05-11T14:30:13,909 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_marine_license.yaml' 2026-05-11T14:30:13,910 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_social_science.yaml' 2026-05-11T14:30:13,911 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_stem.yaml' 2026-05-11T14:30:13,912 adding 'lm_eval/tasks/include/few_shot_og/Russian/utils.py' 2026-05-11T14:30:13,914 adding 'lm_eval/tasks/include/few_shot_og/Serbian/_include_base_44_serbian.yaml' 2026-05-11T14:30:13,915 adding 'lm_eval/tasks/include/few_shot_og/Serbian/_serbian_few_shot_og_template_yaml' 2026-05-11T14:30:13,916 adding 'lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_arts_humanities.yaml' 2026-05-11T14:30:13,918 adding 'lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_social_science.yaml' 2026-05-11T14:30:13,919 adding 'lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_stem.yaml' 2026-05-11T14:30:13,920 adding 'lm_eval/tasks/include/few_shot_og/Serbian/utils.py' 2026-05-11T14:30:13,922 adding 'lm_eval/tasks/include/few_shot_og/Spanish/_include_base_44_spanish.yaml' 2026-05-11T14:30:13,923 adding 'lm_eval/tasks/include/few_shot_og/Spanish/_spanish_few_shot_og_template_yaml' 2026-05-11T14:30:13,924 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_arts_humanities.yaml' 2026-05-11T14:30:13,926 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_health_oriented_education.yaml' 2026-05-11T14:30:13,927 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_social_science.yaml' 2026-05-11T14:30:13,928 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_stem.yaml' 2026-05-11T14:30:13,930 adding 'lm_eval/tasks/include/few_shot_og/Spanish/utils.py' 2026-05-11T14:30:13,931 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/_include_base_44_tagalog.yaml' 2026-05-11T14:30:13,932 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/_tagalog_few_shot_og_template_yaml' 2026-05-11T14:30:13,933 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_arts_humanities.yaml' 2026-05-11T14:30:13,935 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_driving_license.yaml' 2026-05-11T14:30:13,936 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/utils.py' 2026-05-11T14:30:13,938 adding 'lm_eval/tasks/include/few_shot_og/Tamil/_include_base_44_tamil.yaml' 2026-05-11T14:30:13,939 adding 'lm_eval/tasks/include/few_shot_og/Tamil/_tamil_few_shot_og_template_yaml' 2026-05-11T14:30:13,940 adding 'lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_general_knowledge.yaml' 2026-05-11T14:30:13,941 adding 'lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_stem.yaml' 2026-05-11T14:30:13,942 adding 'lm_eval/tasks/include/few_shot_og/Tamil/utils.py' 2026-05-11T14:30:13,944 adding 'lm_eval/tasks/include/few_shot_og/Telugu/_include_base_44_telugu.yaml' 2026-05-11T14:30:13,945 adding 'lm_eval/tasks/include/few_shot_og/Telugu/_telugu_few_shot_og_template_yaml' 2026-05-11T14:30:13,946 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_applied_science.yaml' 2026-05-11T14:30:13,948 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_arts_humanities.yaml' 2026-05-11T14:30:13,949 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_social_science.yaml' 2026-05-11T14:30:13,950 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_stem.yaml' 2026-05-11T14:30:13,951 adding 'lm_eval/tasks/include/few_shot_og/Telugu/utils.py' 2026-05-11T14:30:13,953 adding 'lm_eval/tasks/include/few_shot_og/Turkish/_include_base_44_turkish.yaml' 2026-05-11T14:30:13,954 adding 'lm_eval/tasks/include/few_shot_og/Turkish/_turkish_few_shot_og_template_yaml' 2026-05-11T14:30:13,955 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_arts_humanities.yaml' 2026-05-11T14:30:13,957 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_business_commerce.yaml' 2026-05-11T14:30:13,958 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_social_science.yaml' 2026-05-11T14:30:13,959 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_stem.yaml' 2026-05-11T14:30:13,960 adding 'lm_eval/tasks/include/few_shot_og/Turkish/utils.py' 2026-05-11T14:30:13,962 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/_include_base_44_ukrainian.yaml' 2026-05-11T14:30:13,963 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/_ukrainian_few_shot_og_template_yaml' 2026-05-11T14:30:13,964 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml' 2026-05-11T14:30:13,966 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_social_science.yaml' 2026-05-11T14:30:13,967 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_stem.yaml' 2026-05-11T14:30:13,968 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/utils.py' 2026-05-11T14:30:13,970 adding 'lm_eval/tasks/include/few_shot_og/Urdu/_include_base_44_urdu.yaml' 2026-05-11T14:30:13,971 adding 'lm_eval/tasks/include/few_shot_og/Urdu/_urdu_few_shot_og_template_yaml' 2026-05-11T14:30:13,972 adding 'lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_arts_humanities.yaml' 2026-05-11T14:30:13,974 adding 'lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_health_oriented_education.yaml' 2026-05-11T14:30:13,975 adding 'lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_stem.yaml' 2026-05-11T14:30:13,976 adding 'lm_eval/tasks/include/few_shot_og/Urdu/utils.py' 2026-05-11T14:30:13,978 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/_include_base_44_uzbek.yaml' 2026-05-11T14:30:13,979 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/_uzbek_few_shot_og_template_yaml' 2026-05-11T14:30:13,980 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_arts_humanities.yaml' 2026-05-11T14:30:13,981 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_medical_license.yaml' 2026-05-11T14:30:13,982 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_social_science.yaml' 2026-05-11T14:30:13,983 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_stem.yaml' 2026-05-11T14:30:13,984 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/utils.py' 2026-05-11T14:30:13,986 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/_include_base_44_vietnamese.yaml' 2026-05-11T14:30:13,987 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/_vietnamese_few_shot_og_template_yaml' 2026-05-11T14:30:13,988 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml' 2026-05-11T14:30:13,989 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_social_science.yaml' 2026-05-11T14:30:13,991 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_stem.yaml' 2026-05-11T14:30:13,992 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/utils.py' 2026-05-11T14:30:13,994 adding 'lm_eval/tasks/infinitebench/README.md' 2026-05-11T14:30:13,995 adding 'lm_eval/tasks/infinitebench/_infinitebench_common_yaml' 2026-05-11T14:30:13,996 adding 'lm_eval/tasks/infinitebench/infinitebench.yaml' 2026-05-11T14:30:13,997 adding 'lm_eval/tasks/infinitebench/infinitebench_code_debug.yaml' 2026-05-11T14:30:13,998 adding 'lm_eval/tasks/infinitebench/infinitebench_code_run.yaml' 2026-05-11T14:30:14,000 adding 'lm_eval/tasks/infinitebench/infinitebench_kv_retrieval.yaml' 2026-05-11T14:30:14,001 adding 'lm_eval/tasks/infinitebench/infinitebench_longbook_choice_en.yaml' 2026-05-11T14:30:14,002 adding 'lm_eval/tasks/infinitebench/infinitebench_longbook_qa_chn.yaml' 2026-05-11T14:30:14,003 adding 'lm_eval/tasks/infinitebench/infinitebench_longbook_qa_en.yaml' 2026-05-11T14:30:14,004 adding 'lm_eval/tasks/infinitebench/infinitebench_longbook_sum_en.yaml' 2026-05-11T14:30:14,006 adding 'lm_eval/tasks/infinitebench/infinitebench_longdialogue_qa_en.yaml' 2026-05-11T14:30:14,007 adding 'lm_eval/tasks/infinitebench/infinitebench_math_find.yaml' 2026-05-11T14:30:14,008 adding 'lm_eval/tasks/infinitebench/infinitebench_number_string.yaml' 2026-05-11T14:30:14,009 adding 'lm_eval/tasks/infinitebench/infinitebench_passkey.yaml' 2026-05-11T14:30:14,012 adding 'lm_eval/tasks/infinitebench/utils.py' 2026-05-11T14:30:14,014 adding 'lm_eval/tasks/inverse_scaling/README.md' 2026-05-11T14:30:14,015 adding 'lm_eval/tasks/inverse_scaling/_inverse_scaling_mc_yaml' 2026-05-11T14:30:14,017 adding 'lm_eval/tasks/inverse_scaling/_some_results' 2026-05-11T14:30:14,018 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_hindsight_neglect.yaml' 2026-05-11T14:30:14,019 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_into_the_unknown.yaml' 2026-05-11T14:30:14,020 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_memo_trap.yaml' 2026-05-11T14:30:14,021 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_modus_tollens.yaml' 2026-05-11T14:30:14,023 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_neqa.yaml' 2026-05-11T14:30:14,024 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_pattern_matching_suppression.yaml' 2026-05-11T14:30:14,025 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_quote_repetition.yaml' 2026-05-11T14:30:14,026 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_redefine_math.yaml' 2026-05-11T14:30:14,027 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_repetitive_algebra.yaml' 2026-05-11T14:30:14,028 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_sig_figs.yaml' 2026-05-11T14:30:14,030 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_winobias_antistereotype.yaml' 2026-05-11T14:30:14,032 adding 'lm_eval/tasks/japanese_leaderboard/README.md' 2026-05-11T14:30:14,033 adding 'lm_eval/tasks/japanese_leaderboard/_ja_leaderboard.yaml' 2026-05-11T14:30:14,034 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jaqket_v2.yaml' 2026-05-11T14:30:14,035 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.py' 2026-05-11T14:30:14,036 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.yaml' 2026-05-11T14:30:14,038 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jnli.yaml' 2026-05-11T14:30:14,039 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jsquad.yaml' 2026-05-11T14:30:14,040 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_marc_ja.yaml' 2026-05-11T14:30:14,041 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.py' 2026-05-11T14:30:14,042 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.yaml' 2026-05-11T14:30:14,044 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.py' 2026-05-11T14:30:14,045 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.yaml' 2026-05-11T14:30:14,046 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.py' 2026-05-11T14:30:14,047 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.yaml' 2026-05-11T14:30:14,048 adding 'lm_eval/tasks/japanese_leaderboard/requirements.txt' 2026-05-11T14:30:14,050 adding 'lm_eval/tasks/jfinqa/README.md' 2026-05-11T14:30:14,051 adding 'lm_eval/tasks/jfinqa/_jfinqa.yaml' 2026-05-11T14:30:14,053 adding 'lm_eval/tasks/jfinqa/jfinqa_consistency.yaml' 2026-05-11T14:30:14,054 adding 'lm_eval/tasks/jfinqa/jfinqa_numerical.yaml' 2026-05-11T14:30:14,055 adding 'lm_eval/tasks/jfinqa/jfinqa_temporal.yaml' 2026-05-11T14:30:14,057 adding 'lm_eval/tasks/jfinqa/test_jfinqa_utils.py' 2026-05-11T14:30:14,058 adding 'lm_eval/tasks/jfinqa/utils.py' 2026-05-11T14:30:14,060 adding 'lm_eval/tasks/jsonschema_bench/README.md' 2026-05-11T14:30:14,062 adding 'lm_eval/tasks/jsonschema_bench/jsonschema_bench_easy.yaml' 2026-05-11T14:30:14,063 adding 'lm_eval/tasks/jsonschema_bench/jsonschema_bench_hard.yaml' 2026-05-11T14:30:14,064 adding 'lm_eval/tasks/jsonschema_bench/jsonschema_bench_medium.yaml' 2026-05-11T14:30:14,066 adding 'lm_eval/tasks/jsonschema_bench/metrics.py' 2026-05-11T14:30:14,068 adding 'lm_eval/tasks/kbl/README.md' 2026-05-11T14:30:14,070 adding 'lm_eval/tasks/kbl/bar_exam/civil/_base_em_yaml' 2026-05-11T14:30:14,071 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2012.yaml' 2026-05-11T14:30:14,073 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2013.yaml' 2026-05-11T14:30:14,074 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2014.yaml' 2026-05-11T14:30:14,075 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2015.yaml' 2026-05-11T14:30:14,076 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2016.yaml' 2026-05-11T14:30:14,077 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2017.yaml' 2026-05-11T14:30:14,078 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2018.yaml' 2026-05-11T14:30:14,080 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2019.yaml' 2026-05-11T14:30:14,081 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2020.yaml' 2026-05-11T14:30:14,082 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2021.yaml' 2026-05-11T14:30:14,083 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2022.yaml' 2026-05-11T14:30:14,084 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2023.yaml' 2026-05-11T14:30:14,085 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2024.yaml' 2026-05-11T14:30:14,086 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2025.yaml' 2026-05-11T14:30:14,089 adding 'lm_eval/tasks/kbl/bar_exam/criminal/_base_em_yaml' 2026-05-11T14:30:14,090 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2012.yaml' 2026-05-11T14:30:14,091 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2013.yaml' 2026-05-11T14:30:14,092 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2014.yaml' 2026-05-11T14:30:14,093 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2015.yaml' 2026-05-11T14:30:14,094 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2016.yaml' 2026-05-11T14:30:14,095 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2017.yaml' 2026-05-11T14:30:14,096 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2018.yaml' 2026-05-11T14:30:14,098 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2019.yaml' 2026-05-11T14:30:14,099 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2020.yaml' 2026-05-11T14:30:14,100 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2021.yaml' 2026-05-11T14:30:14,101 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2022.yaml' 2026-05-11T14:30:14,102 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2023.yaml' 2026-05-11T14:30:14,103 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2024.yaml' 2026-05-11T14:30:14,105 adding 'lm_eval/tasks/kbl/bar_exam/public/_base_em_yaml' 2026-05-11T14:30:14,106 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2012.yaml' 2026-05-11T14:30:14,107 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2013.yaml' 2026-05-11T14:30:14,108 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2014.yaml' 2026-05-11T14:30:14,110 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2015.yaml' 2026-05-11T14:30:14,111 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2016.yaml' 2026-05-11T14:30:14,112 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2017.yaml' 2026-05-11T14:30:14,113 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2018.yaml' 2026-05-11T14:30:14,114 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2019.yaml' 2026-05-11T14:30:14,115 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2020.yaml' 2026-05-11T14:30:14,116 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2021.yaml' 2026-05-11T14:30:14,117 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2022.yaml' 2026-05-11T14:30:14,119 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2023.yaml' 2026-05-11T14:30:14,120 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2024.yaml' 2026-05-11T14:30:14,122 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/_base_em_yaml' 2026-05-11T14:30:14,123 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2010.yaml' 2026-05-11T14:30:14,124 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2011.yaml' 2026-05-11T14:30:14,125 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2012.yaml' 2026-05-11T14:30:14,127 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2013.yaml' 2026-05-11T14:30:14,128 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2014.yaml' 2026-05-11T14:30:14,129 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2015.yaml' 2026-05-11T14:30:14,130 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2016.yaml' 2026-05-11T14:30:14,131 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2017.yaml' 2026-05-11T14:30:14,132 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2018.yaml' 2026-05-11T14:30:14,133 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2019.yaml' 2026-05-11T14:30:14,135 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2020.yaml' 2026-05-11T14:30:14,136 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2021.yaml' 2026-05-11T14:30:14,137 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2022.yaml' 2026-05-11T14:30:14,138 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2023.yaml' 2026-05-11T14:30:14,140 adding 'lm_eval/tasks/kbl/knowledge/_kbl_knowledge_yaml' 2026-05-11T14:30:14,141 adding 'lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_em.yaml' 2026-05-11T14:30:14,142 adding 'lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_reasoning_em.yaml' 2026-05-11T14:30:14,143 adding 'lm_eval/tasks/kbl/knowledge/kbl_legal_concept_qa_em.yaml' 2026-05-11T14:30:14,144 adding 'lm_eval/tasks/kbl/knowledge/kbl_offense_component_qa_em.yaml' 2026-05-11T14:30:14,145 adding 'lm_eval/tasks/kbl/knowledge/kbl_query_statute_matching_qa_em.yaml' 2026-05-11T14:30:14,147 adding 'lm_eval/tasks/kbl/knowledge/kbl_statute_hallucination_qa_em.yaml' 2026-05-11T14:30:14,148 adding 'lm_eval/tasks/kbl/knowledge/kbl_statute_number_and_content_matching_qa_em.yaml' 2026-05-11T14:30:14,150 adding 'lm_eval/tasks/kbl/reasoning/_kbl_reasoning_yaml' 2026-05-11T14:30:14,151 adding 'lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_p_em.yaml' 2026-05-11T14:30:14,152 adding 'lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_q_em.yaml' 2026-05-11T14:30:14,153 adding 'lm_eval/tasks/kbl/reasoning/kbl_causal_reasoning_em.yaml' 2026-05-11T14:30:14,154 adding 'lm_eval/tasks/kbl/reasoning/kbl_statement_consistency_qa_em.yaml' 2026-05-11T14:30:14,156 adding 'lm_eval/tasks/kmmlu/README.md' 2026-05-11T14:30:14,159 adding 'lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml' 2026-05-11T14:30:14,160 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard.yaml' 2026-05-11T14:30:14,161 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_applied_science.yaml' 2026-05-11T14:30:14,163 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_humss.yaml' 2026-05-11T14:30:14,164 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_other.yaml' 2026-05-11T14:30:14,165 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_stem.yaml' 2026-05-11T14:30:14,167 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml' 2026-05-11T14:30:14,169 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml' 2026-05-11T14:30:14,170 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml' 2026-05-11T14:30:14,172 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml' 2026-05-11T14:30:14,174 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml' 2026-05-11T14:30:14,176 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml' 2026-05-11T14:30:14,181 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml' 2026-05-11T14:30:14,183 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml' 2026-05-11T14:30:14,185 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml' 2026-05-11T14:30:14,187 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml' 2026-05-11T14:30:14,189 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml' 2026-05-11T14:30:14,191 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml' 2026-05-11T14:30:14,193 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml' 2026-05-11T14:30:14,195 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml' 2026-05-11T14:30:14,197 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml' 2026-05-11T14:30:14,199 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml' 2026-05-11T14:30:14,200 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml' 2026-05-11T14:30:14,202 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml' 2026-05-11T14:30:14,204 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml' 2026-05-11T14:30:14,206 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml' 2026-05-11T14:30:14,208 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml' 2026-05-11T14:30:14,210 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml' 2026-05-11T14:30:14,211 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml' 2026-05-11T14:30:14,213 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml' 2026-05-11T14:30:14,215 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml' 2026-05-11T14:30:14,217 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml' 2026-05-11T14:30:14,219 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml' 2026-05-11T14:30:14,221 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml' 2026-05-11T14:30:14,222 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml' 2026-05-11T14:30:14,224 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml' 2026-05-11T14:30:14,226 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml' 2026-05-11T14:30:14,228 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml' 2026-05-11T14:30:14,230 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml' 2026-05-11T14:30:14,232 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml' 2026-05-11T14:30:14,234 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml' 2026-05-11T14:30:14,236 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml' 2026-05-11T14:30:14,238 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml' 2026-05-11T14:30:14,240 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml' 2026-05-11T14:30:14,242 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml' 2026-05-11T14:30:14,244 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml' 2026-05-11T14:30:14,246 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml' 2026-05-11T14:30:14,248 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml' 2026-05-11T14:30:14,249 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml' 2026-05-11T14:30:14,252 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml' 2026-05-11T14:30:14,253 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml' 2026-05-11T14:30:14,256 adding 'lm_eval/tasks/kmmlu/default/_default_kmmlu_yaml' 2026-05-11T14:30:14,257 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_applied_science.yaml' 2026-05-11T14:30:14,259 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_default.yaml' 2026-05-11T14:30:14,260 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_humss.yaml' 2026-05-11T14:30:14,261 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_other.yaml' 2026-05-11T14:30:14,262 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_stem.yaml' 2026-05-11T14:30:14,263 adding 'lm_eval/tasks/kmmlu/default/kmmlu_accounting.yaml' 2026-05-11T14:30:14,264 adding 'lm_eval/tasks/kmmlu/default/kmmlu_agricultural_sciences.yaml' 2026-05-11T14:30:14,266 adding 'lm_eval/tasks/kmmlu/default/kmmlu_aviation_engineering_and_maintenance.yaml' 2026-05-11T14:30:14,267 adding 'lm_eval/tasks/kmmlu/default/kmmlu_biology.yaml' 2026-05-11T14:30:14,268 adding 'lm_eval/tasks/kmmlu/default/kmmlu_chemical_engineering.yaml' 2026-05-11T14:30:14,269 adding 'lm_eval/tasks/kmmlu/default/kmmlu_chemistry.yaml' 2026-05-11T14:30:14,270 adding 'lm_eval/tasks/kmmlu/default/kmmlu_civil_engineering.yaml' 2026-05-11T14:30:14,271 adding 'lm_eval/tasks/kmmlu/default/kmmlu_computer_science.yaml' 2026-05-11T14:30:14,273 adding 'lm_eval/tasks/kmmlu/default/kmmlu_construction.yaml' 2026-05-11T14:30:14,274 adding 'lm_eval/tasks/kmmlu/default/kmmlu_criminal_law.yaml' 2026-05-11T14:30:14,275 adding 'lm_eval/tasks/kmmlu/default/kmmlu_ecology.yaml' 2026-05-11T14:30:14,276 adding 'lm_eval/tasks/kmmlu/default/kmmlu_economics.yaml' 2026-05-11T14:30:14,277 adding 'lm_eval/tasks/kmmlu/default/kmmlu_education.yaml' 2026-05-11T14:30:14,278 adding 'lm_eval/tasks/kmmlu/default/kmmlu_electrical_engineering.yaml' 2026-05-11T14:30:14,279 adding 'lm_eval/tasks/kmmlu/default/kmmlu_electronics_engineering.yaml' 2026-05-11T14:30:14,280 adding 'lm_eval/tasks/kmmlu/default/kmmlu_energy_management.yaml' 2026-05-11T14:30:14,281 adding 'lm_eval/tasks/kmmlu/default/kmmlu_environmental_science.yaml' 2026-05-11T14:30:14,282 adding 'lm_eval/tasks/kmmlu/default/kmmlu_fashion.yaml' 2026-05-11T14:30:14,283 adding 'lm_eval/tasks/kmmlu/default/kmmlu_food_processing.yaml' 2026-05-11T14:30:14,284 adding 'lm_eval/tasks/kmmlu/default/kmmlu_gas_technology_and_engineering.yaml' 2026-05-11T14:30:14,285 adding 'lm_eval/tasks/kmmlu/default/kmmlu_geomatics.yaml' 2026-05-11T14:30:14,286 adding 'lm_eval/tasks/kmmlu/default/kmmlu_health.yaml' 2026-05-11T14:30:14,287 adding 'lm_eval/tasks/kmmlu/default/kmmlu_industrial_engineer.yaml' 2026-05-11T14:30:14,289 adding 'lm_eval/tasks/kmmlu/default/kmmlu_information_technology.yaml' 2026-05-11T14:30:14,290 adding 'lm_eval/tasks/kmmlu/default/kmmlu_interior_architecture_and_design.yaml' 2026-05-11T14:30:14,291 adding 'lm_eval/tasks/kmmlu/default/kmmlu_korean_history.yaml' 2026-05-11T14:30:14,292 adding 'lm_eval/tasks/kmmlu/default/kmmlu_law.yaml' 2026-05-11T14:30:14,293 adding 'lm_eval/tasks/kmmlu/default/kmmlu_machine_design_and_manufacturing.yaml' 2026-05-11T14:30:14,294 adding 'lm_eval/tasks/kmmlu/default/kmmlu_management.yaml' 2026-05-11T14:30:14,295 adding 'lm_eval/tasks/kmmlu/default/kmmlu_maritime_engineering.yaml' 2026-05-11T14:30:14,296 adding 'lm_eval/tasks/kmmlu/default/kmmlu_marketing.yaml' 2026-05-11T14:30:14,297 adding 'lm_eval/tasks/kmmlu/default/kmmlu_materials_engineering.yaml' 2026-05-11T14:30:14,299 adding 'lm_eval/tasks/kmmlu/default/kmmlu_math.yaml' 2026-05-11T14:30:14,300 adding 'lm_eval/tasks/kmmlu/default/kmmlu_mechanical_engineering.yaml' 2026-05-11T14:30:14,301 adding 'lm_eval/tasks/kmmlu/default/kmmlu_nondestructive_testing.yaml' 2026-05-11T14:30:14,302 adding 'lm_eval/tasks/kmmlu/default/kmmlu_patent.yaml' 2026-05-11T14:30:14,303 adding 'lm_eval/tasks/kmmlu/default/kmmlu_political_science_and_sociology.yaml' 2026-05-11T14:30:14,304 adding 'lm_eval/tasks/kmmlu/default/kmmlu_psychology.yaml' 2026-05-11T14:30:14,305 adding 'lm_eval/tasks/kmmlu/default/kmmlu_public_safety.yaml' 2026-05-11T14:30:14,306 adding 'lm_eval/tasks/kmmlu/default/kmmlu_railway_and_automotive_engineering.yaml' 2026-05-11T14:30:14,308 adding 'lm_eval/tasks/kmmlu/default/kmmlu_real_estate.yaml' 2026-05-11T14:30:14,309 adding 'lm_eval/tasks/kmmlu/default/kmmlu_refrigerating_machinery.yaml' 2026-05-11T14:30:14,310 adding 'lm_eval/tasks/kmmlu/default/kmmlu_social_welfare.yaml' 2026-05-11T14:30:14,311 adding 'lm_eval/tasks/kmmlu/default/kmmlu_taxation.yaml' 2026-05-11T14:30:14,312 adding 'lm_eval/tasks/kmmlu/default/kmmlu_telecommunications_and_wireless_technology.yaml' 2026-05-11T14:30:14,315 adding 'lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml' 2026-05-11T14:30:14,316 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct.yaml' 2026-05-11T14:30:14,317 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_applied_science.yaml' 2026-05-11T14:30:14,318 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_humss.yaml' 2026-05-11T14:30:14,319 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_other.yaml' 2026-05-11T14:30:14,321 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_stem.yaml' 2026-05-11T14:30:14,322 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml' 2026-05-11T14:30:14,323 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml' 2026-05-11T14:30:14,324 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml' 2026-05-11T14:30:14,325 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml' 2026-05-11T14:30:14,326 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml' 2026-05-11T14:30:14,327 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml' 2026-05-11T14:30:14,328 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml' 2026-05-11T14:30:14,329 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml' 2026-05-11T14:30:14,330 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml' 2026-05-11T14:30:14,331 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml' 2026-05-11T14:30:14,332 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml' 2026-05-11T14:30:14,333 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml' 2026-05-11T14:30:14,335 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml' 2026-05-11T14:30:14,336 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml' 2026-05-11T14:30:14,337 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml' 2026-05-11T14:30:14,338 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml' 2026-05-11T14:30:14,339 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml' 2026-05-11T14:30:14,340 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml' 2026-05-11T14:30:14,341 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml' 2026-05-11T14:30:14,343 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml' 2026-05-11T14:30:14,344 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml' 2026-05-11T14:30:14,345 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml' 2026-05-11T14:30:14,346 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml' 2026-05-11T14:30:14,347 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml' 2026-05-11T14:30:14,348 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml' 2026-05-11T14:30:14,349 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml' 2026-05-11T14:30:14,351 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml' 2026-05-11T14:30:14,352 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml' 2026-05-11T14:30:14,353 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml' 2026-05-11T14:30:14,354 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml' 2026-05-11T14:30:14,355 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml' 2026-05-11T14:30:14,356 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml' 2026-05-11T14:30:14,357 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml' 2026-05-11T14:30:14,359 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml' 2026-05-11T14:30:14,360 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml' 2026-05-11T14:30:14,361 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml' 2026-05-11T14:30:14,362 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml' 2026-05-11T14:30:14,363 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml' 2026-05-11T14:30:14,365 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml' 2026-05-11T14:30:14,366 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml' 2026-05-11T14:30:14,367 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml' 2026-05-11T14:30:14,368 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml' 2026-05-11T14:30:14,369 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml' 2026-05-11T14:30:14,370 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml' 2026-05-11T14:30:14,372 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml' 2026-05-11T14:30:14,374 adding 'lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml' 2026-05-11T14:30:14,376 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard.yaml' 2026-05-11T14:30:14,377 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_applied_science.yaml' 2026-05-11T14:30:14,378 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_humss.yaml' 2026-05-11T14:30:14,379 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_other.yaml' 2026-05-11T14:30:14,380 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_stem.yaml' 2026-05-11T14:30:14,381 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml' 2026-05-11T14:30:14,382 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml' 2026-05-11T14:30:14,383 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml' 2026-05-11T14:30:14,384 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml' 2026-05-11T14:30:14,385 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml' 2026-05-11T14:30:14,386 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml' 2026-05-11T14:30:14,388 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml' 2026-05-11T14:30:14,389 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml' 2026-05-11T14:30:14,390 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml' 2026-05-11T14:30:14,391 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml' 2026-05-11T14:30:14,392 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml' 2026-05-11T14:30:14,393 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml' 2026-05-11T14:30:14,394 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml' 2026-05-11T14:30:14,395 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml' 2026-05-11T14:30:14,397 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml' 2026-05-11T14:30:14,398 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml' 2026-05-11T14:30:14,399 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml' 2026-05-11T14:30:14,400 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml' 2026-05-11T14:30:14,401 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml' 2026-05-11T14:30:14,402 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml' 2026-05-11T14:30:14,404 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml' 2026-05-11T14:30:14,405 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml' 2026-05-11T14:30:14,406 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml' 2026-05-11T14:30:14,407 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml' 2026-05-11T14:30:14,408 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml' 2026-05-11T14:30:14,409 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml' 2026-05-11T14:30:14,410 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml' 2026-05-11T14:30:14,412 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml' 2026-05-11T14:30:14,413 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml' 2026-05-11T14:30:14,414 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml' 2026-05-11T14:30:14,415 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml' 2026-05-11T14:30:14,416 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml' 2026-05-11T14:30:14,417 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml' 2026-05-11T14:30:14,419 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml' 2026-05-11T14:30:14,420 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml' 2026-05-11T14:30:14,421 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml' 2026-05-11T14:30:14,422 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml' 2026-05-11T14:30:14,423 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml' 2026-05-11T14:30:14,424 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml' 2026-05-11T14:30:14,425 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml' 2026-05-11T14:30:14,426 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml' 2026-05-11T14:30:14,427 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml' 2026-05-11T14:30:14,428 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml' 2026-05-11T14:30:14,429 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml' 2026-05-11T14:30:14,431 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml' 2026-05-11T14:30:14,433 adding 'lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml' 2026-05-11T14:30:14,434 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard.yaml' 2026-05-11T14:30:14,436 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_applied_science.yaml' 2026-05-11T14:30:14,437 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_humss.yaml' 2026-05-11T14:30:14,438 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_other.yaml' 2026-05-11T14:30:14,439 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_stem.yaml' 2026-05-11T14:30:14,440 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml' 2026-05-11T14:30:14,441 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml' 2026-05-11T14:30:14,442 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml' 2026-05-11T14:30:14,444 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml' 2026-05-11T14:30:14,445 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml' 2026-05-11T14:30:14,446 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml' 2026-05-11T14:30:14,447 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml' 2026-05-11T14:30:14,448 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml' 2026-05-11T14:30:14,449 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml' 2026-05-11T14:30:14,450 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml' 2026-05-11T14:30:14,451 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml' 2026-05-11T14:30:14,453 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml' 2026-05-11T14:30:14,454 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml' 2026-05-11T14:30:14,455 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml' 2026-05-11T14:30:14,456 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml' 2026-05-11T14:30:14,457 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml' 2026-05-11T14:30:14,459 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml' 2026-05-11T14:30:14,460 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml' 2026-05-11T14:30:14,461 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml' 2026-05-11T14:30:14,462 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml' 2026-05-11T14:30:14,463 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml' 2026-05-11T14:30:14,464 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml' 2026-05-11T14:30:14,466 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml' 2026-05-11T14:30:14,467 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml' 2026-05-11T14:30:14,468 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml' 2026-05-11T14:30:14,469 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml' 2026-05-11T14:30:14,470 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml' 2026-05-11T14:30:14,471 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml' 2026-05-11T14:30:14,472 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml' 2026-05-11T14:30:14,473 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml' 2026-05-11T14:30:14,474 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml' 2026-05-11T14:30:14,475 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml' 2026-05-11T14:30:14,476 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml' 2026-05-11T14:30:14,477 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml' 2026-05-11T14:30:14,479 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml' 2026-05-11T14:30:14,480 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml' 2026-05-11T14:30:14,481 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml' 2026-05-11T14:30:14,482 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml' 2026-05-11T14:30:14,483 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml' 2026-05-11T14:30:14,484 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml' 2026-05-11T14:30:14,485 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml' 2026-05-11T14:30:14,487 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml' 2026-05-11T14:30:14,488 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml' 2026-05-11T14:30:14,489 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml' 2026-05-11T14:30:14,491 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml' 2026-05-11T14:30:14,493 adding 'lm_eval/tasks/kobest/README.md' 2026-05-11T14:30:14,494 adding 'lm_eval/tasks/kobest/_kobest.yaml' 2026-05-11T14:30:14,495 adding 'lm_eval/tasks/kobest/kobest_boolq.yaml' 2026-05-11T14:30:14,496 adding 'lm_eval/tasks/kobest/kobest_copa.yaml' 2026-05-11T14:30:14,498 adding 'lm_eval/tasks/kobest/kobest_hellaswag.yaml' 2026-05-11T14:30:14,499 adding 'lm_eval/tasks/kobest/kobest_sentineg.yaml' 2026-05-11T14:30:14,500 adding 'lm_eval/tasks/kobest/kobest_wic.yaml' 2026-05-11T14:30:14,502 adding 'lm_eval/tasks/kobest/utils.py' 2026-05-11T14:30:14,504 adding 'lm_eval/tasks/kormedmcqa/README.md' 2026-05-11T14:30:14,505 adding 'lm_eval/tasks/kormedmcqa/_kormedmcqa.yaml' 2026-05-11T14:30:14,506 adding 'lm_eval/tasks/kormedmcqa/_template_yaml' 2026-05-11T14:30:14,507 adding 'lm_eval/tasks/kormedmcqa/dentist.yaml' 2026-05-11T14:30:14,508 adding 'lm_eval/tasks/kormedmcqa/doctor.yaml' 2026-05-11T14:30:14,510 adding 'lm_eval/tasks/kormedmcqa/nurse.yaml' 2026-05-11T14:30:14,511 adding 'lm_eval/tasks/kormedmcqa/pharm.yaml' 2026-05-11T14:30:14,512 adding 'lm_eval/tasks/lambada/README.md' 2026-05-11T14:30:14,514 adding 'lm_eval/tasks/lambada/lambada_openai.yaml' 2026-05-11T14:30:14,515 adding 'lm_eval/tasks/lambada/lambada_standard.yaml' 2026-05-11T14:30:14,517 adding 'lm_eval/tasks/lambada_cloze/README.md' 2026-05-11T14:30:14,518 adding 'lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml' 2026-05-11T14:30:14,519 adding 'lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml' 2026-05-11T14:30:14,521 adding 'lm_eval/tasks/lambada_multilingual/README.md' 2026-05-11T14:30:14,522 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml' 2026-05-11T14:30:14,523 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml' 2026-05-11T14:30:14,525 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml' 2026-05-11T14:30:14,526 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml' 2026-05-11T14:30:14,527 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml' 2026-05-11T14:30:14,529 adding 'lm_eval/tasks/lambada_multilingual_stablelm/README.md' 2026-05-11T14:30:14,530 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml' 2026-05-11T14:30:14,531 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml' 2026-05-11T14:30:14,532 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml' 2026-05-11T14:30:14,533 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml' 2026-05-11T14:30:14,534 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml' 2026-05-11T14:30:14,535 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml' 2026-05-11T14:30:14,536 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml' 2026-05-11T14:30:14,539 adding 'lm_eval/tasks/leaderboard/README.md' 2026-05-11T14:30:14,540 adding 'lm_eval/tasks/leaderboard/leaderboard.yaml' 2026-05-11T14:30:14,542 adding 'lm_eval/tasks/leaderboard/bbh_mc/_fewshot_template_yaml' 2026-05-11T14:30:14,543 adding 'lm_eval/tasks/leaderboard/bbh_mc/_leaderboard_bbh.yaml' 2026-05-11T14:30:14,544 adding 'lm_eval/tasks/leaderboard/bbh_mc/boolean_expressions.yaml' 2026-05-11T14:30:14,546 adding 'lm_eval/tasks/leaderboard/bbh_mc/causal_judgement.yaml' 2026-05-11T14:30:14,547 adding 'lm_eval/tasks/leaderboard/bbh_mc/date_understanding.yaml' 2026-05-11T14:30:14,548 adding 'lm_eval/tasks/leaderboard/bbh_mc/disambiguation_qa.yaml' 2026-05-11T14:30:14,550 adding 'lm_eval/tasks/leaderboard/bbh_mc/formal_fallacies.yaml' 2026-05-11T14:30:14,551 adding 'lm_eval/tasks/leaderboard/bbh_mc/geometric_shapes.yaml' 2026-05-11T14:30:14,552 adding 'lm_eval/tasks/leaderboard/bbh_mc/hyperbaton.yaml' 2026-05-11T14:30:14,553 adding 'lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_five_objects.yaml' 2026-05-11T14:30:14,554 adding 'lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_seven_objects.yaml' 2026-05-11T14:30:14,556 adding 'lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_three_objects.yaml' 2026-05-11T14:30:14,557 adding 'lm_eval/tasks/leaderboard/bbh_mc/movie_recommendation.yaml' 2026-05-11T14:30:14,558 adding 'lm_eval/tasks/leaderboard/bbh_mc/navigate.yaml' 2026-05-11T14:30:14,559 adding 'lm_eval/tasks/leaderboard/bbh_mc/object_counting.yaml' 2026-05-11T14:30:14,561 adding 'lm_eval/tasks/leaderboard/bbh_mc/penguins_in_a_table.yaml' 2026-05-11T14:30:14,562 adding 'lm_eval/tasks/leaderboard/bbh_mc/reasoning_about_colored_objects.yaml' 2026-05-11T14:30:14,563 adding 'lm_eval/tasks/leaderboard/bbh_mc/ruin_names.yaml' 2026-05-11T14:30:14,564 adding 'lm_eval/tasks/leaderboard/bbh_mc/salient_translation_error_detection.yaml' 2026-05-11T14:30:14,566 adding 'lm_eval/tasks/leaderboard/bbh_mc/snarks.yaml' 2026-05-11T14:30:14,567 adding 'lm_eval/tasks/leaderboard/bbh_mc/sports_understanding.yaml' 2026-05-11T14:30:14,568 adding 'lm_eval/tasks/leaderboard/bbh_mc/temporal_sequences.yaml' 2026-05-11T14:30:14,569 adding 'lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T14:30:14,571 adding 'lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T14:30:14,572 adding 'lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T14:30:14,573 adding 'lm_eval/tasks/leaderboard/bbh_mc/web_of_lies.yaml' 2026-05-11T14:30:14,575 adding 'lm_eval/tasks/leaderboard/gpqa/_leaderboard_gpqa.yaml' 2026-05-11T14:30:14,576 adding 'lm_eval/tasks/leaderboard/gpqa/_template_yaml' 2026-05-11T14:30:14,577 adding 'lm_eval/tasks/leaderboard/gpqa/gpqa_diamond_zeroshot.yaml' 2026-05-11T14:30:14,579 adding 'lm_eval/tasks/leaderboard/gpqa/gpqa_extended_zeroshot.yaml' 2026-05-11T14:30:14,580 adding 'lm_eval/tasks/leaderboard/gpqa/gpqa_main_zeroshot.yaml' 2026-05-11T14:30:14,581 adding 'lm_eval/tasks/leaderboard/gpqa/utils.py' 2026-05-11T14:30:14,583 adding 'lm_eval/tasks/leaderboard/ifeval/_leaderboard_instruction_following.yaml' 2026-05-11T14:30:14,584 adding 'lm_eval/tasks/leaderboard/ifeval/ifeval.yaml' 2026-05-11T14:30:14,589 adding 'lm_eval/tasks/leaderboard/ifeval/instructions.py' 2026-05-11T14:30:14,591 adding 'lm_eval/tasks/leaderboard/ifeval/instructions_registry.py' 2026-05-11T14:30:14,594 adding 'lm_eval/tasks/leaderboard/ifeval/instructions_util.py' 2026-05-11T14:30:14,595 adding 'lm_eval/tasks/leaderboard/ifeval/utils.py' 2026-05-11T14:30:14,597 adding 'lm_eval/tasks/leaderboard/math/_leaderboard_math.yaml' 2026-05-11T14:30:14,598 adding 'lm_eval/tasks/leaderboard/math/_template_yaml' 2026-05-11T14:30:14,599 adding 'lm_eval/tasks/leaderboard/math/math_algebra.yaml' 2026-05-11T14:30:14,600 adding 'lm_eval/tasks/leaderboard/math/math_counting_and_prob.yaml' 2026-05-11T14:30:14,601 adding 'lm_eval/tasks/leaderboard/math/math_geometry.yaml' 2026-05-11T14:30:14,602 adding 'lm_eval/tasks/leaderboard/math/math_intermediate_algebra.yaml' 2026-05-11T14:30:14,603 adding 'lm_eval/tasks/leaderboard/math/math_num_theory.yaml' 2026-05-11T14:30:14,605 adding 'lm_eval/tasks/leaderboard/math/math_prealgebra.yaml' 2026-05-11T14:30:14,606 adding 'lm_eval/tasks/leaderboard/math/math_precalculus.yaml' 2026-05-11T14:30:14,608 adding 'lm_eval/tasks/leaderboard/math/utils.py' 2026-05-11T14:30:14,609 adding 'lm_eval/tasks/leaderboard/mmlu_pro/mmlu_pro.yaml' 2026-05-11T14:30:14,610 adding 'lm_eval/tasks/leaderboard/mmlu_pro/utils.py' 2026-05-11T14:30:14,612 adding 'lm_eval/tasks/leaderboard/musr/_musr.yaml' 2026-05-11T14:30:14,613 adding 'lm_eval/tasks/leaderboard/musr/_template_yaml' 2026-05-11T14:30:14,614 adding 'lm_eval/tasks/leaderboard/musr/musr_murder_mysteries.yaml' 2026-05-11T14:30:14,615 adding 'lm_eval/tasks/leaderboard/musr/musr_object_placements.yaml' 2026-05-11T14:30:14,616 adding 'lm_eval/tasks/leaderboard/musr/musr_team_allocation.yaml' 2026-05-11T14:30:14,617 adding 'lm_eval/tasks/leaderboard/musr/utils.py' 2026-05-11T14:30:14,620 adding 'lm_eval/tasks/libra/README.md' 2026-05-11T14:30:14,621 adding 'lm_eval/tasks/libra/_complex_reasoning_and_mathematical_problems.yaml' 2026-05-11T14:30:14,622 adding 'lm_eval/tasks/libra/_multi_hop_question_answering.yaml' 2026-05-11T14:30:14,623 adding 'lm_eval/tasks/libra/_question_answering_and_multiple_choice.yaml' 2026-05-11T14:30:14,624 adding 'lm_eval/tasks/libra/_simple_information_retrieval.yaml' 2026-05-11T14:30:14,625 adding 'lm_eval/tasks/libra/_template_yaml' 2026-05-11T14:30:14,627 adding 'lm_eval/tasks/libra/librusec_history.yaml' 2026-05-11T14:30:14,628 adding 'lm_eval/tasks/libra/librusec_mhqa.yaml' 2026-05-11T14:30:14,629 adding 'lm_eval/tasks/libra/long_context_multiq.yaml' 2026-05-11T14:30:14,630 adding 'lm_eval/tasks/libra/matreshka_names.yaml' 2026-05-11T14:30:14,632 adding 'lm_eval/tasks/libra/matreshka_yes_no.yaml' 2026-05-11T14:30:14,633 adding 'lm_eval/tasks/libra/passkey.yaml' 2026-05-11T14:30:14,634 adding 'lm_eval/tasks/libra/passkey_with_librusec.yaml' 2026-05-11T14:30:14,635 adding 'lm_eval/tasks/libra/ru_2wikimultihopqa.yaml' 2026-05-11T14:30:14,637 adding 'lm_eval/tasks/libra/ru_babilong_qa1.yaml' 2026-05-11T14:30:14,638 adding 'lm_eval/tasks/libra/ru_babilong_qa2.yaml' 2026-05-11T14:30:14,639 adding 'lm_eval/tasks/libra/ru_babilong_qa3.yaml' 2026-05-11T14:30:14,641 adding 'lm_eval/tasks/libra/ru_babilong_qa4.yaml' 2026-05-11T14:30:14,642 adding 'lm_eval/tasks/libra/ru_babilong_qa5.yaml' 2026-05-11T14:30:14,643 adding 'lm_eval/tasks/libra/ru_gsm100.yaml' 2026-05-11T14:30:14,644 adding 'lm_eval/tasks/libra/ru_qasper.yaml' 2026-05-11T14:30:14,645 adding 'lm_eval/tasks/libra/ru_quality.yaml' 2026-05-11T14:30:14,647 adding 'lm_eval/tasks/libra/ru_sci_abstract_retrieval.yaml' 2026-05-11T14:30:14,648 adding 'lm_eval/tasks/libra/ru_sci_passage_count.yaml' 2026-05-11T14:30:14,650 adding 'lm_eval/tasks/libra/utils.py' 2026-05-11T14:30:14,652 adding 'lm_eval/tasks/lingoly/README.md' 2026-05-11T14:30:14,653 adding 'lm_eval/tasks/lingoly/lingoly_context.yaml' 2026-05-11T14:30:14,654 adding 'lm_eval/tasks/lingoly/lingoly_group.yaml' 2026-05-11T14:30:14,655 adding 'lm_eval/tasks/lingoly/lingoly_nocontext.yaml' 2026-05-11T14:30:14,657 adding 'lm_eval/tasks/lingoly/script.py' 2026-05-11T14:30:14,658 adding 'lm_eval/tasks/lingoly/utils.py' 2026-05-11T14:30:14,660 adding 'lm_eval/tasks/llama3/README.md' 2026-05-11T14:30:14,662 adding 'lm_eval/tasks/llama3/instruct/arc_challenge/arc_challenge_llama.yaml' 2026-05-11T14:30:14,663 adding 'lm_eval/tasks/llama3/instruct/arc_challenge/utils.py' 2026-05-11T14:30:14,665 adding 'lm_eval/tasks/llama3/instruct/gsm8k/gsm8k.yaml' 2026-05-11T14:30:14,667 adding 'lm_eval/tasks/llama3/instruct/mmlu/_continuation_template_yaml' 2026-05-11T14:30:14,668 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_humanities.yaml' 2026-05-11T14:30:14,669 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_other.yaml' 2026-05-11T14:30:14,671 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_social_sciences.yaml' 2026-05-11T14:30:14,672 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_stem.yaml' 2026-05-11T14:30:14,673 adding 'lm_eval/tasks/llama3/instruct/mmlu/llama.yaml' 2026-05-11T14:30:14,674 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:14,675 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_anatomy.yaml' 2026-05-11T14:30:14,676 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_astronomy.yaml' 2026-05-11T14:30:14,677 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_business_ethics.yaml' 2026-05-11T14:30:14,678 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:14,679 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_biology.yaml' 2026-05-11T14:30:14,681 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_chemistry.yaml' 2026-05-11T14:30:14,682 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_computer_science.yaml' 2026-05-11T14:30:14,683 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_mathematics.yaml' 2026-05-11T14:30:14,684 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_medicine.yaml' 2026-05-11T14:30:14,685 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_physics.yaml' 2026-05-11T14:30:14,686 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_computer_security.yaml' 2026-05-11T14:30:14,687 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:14,689 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_econometrics.yaml' 2026-05-11T14:30:14,690 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:14,691 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:14,692 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_formal_logic.yaml' 2026-05-11T14:30:14,693 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_global_facts.yaml' 2026-05-11T14:30:14,694 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_biology.yaml' 2026-05-11T14:30:14,695 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:14,697 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:14,698 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:14,699 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_geography.yaml' 2026-05-11T14:30:14,700 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:14,701 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:14,703 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:14,704 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:14,705 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_physics.yaml' 2026-05-11T14:30:14,706 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:14,707 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:14,708 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:14,710 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:14,711 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_aging.yaml' 2026-05-11T14:30:14,712 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_sexuality.yaml' 2026-05-11T14:30:14,713 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_international_law.yaml' 2026-05-11T14:30:14,714 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_jurisprudence.yaml' 2026-05-11T14:30:14,715 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:14,716 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_machine_learning.yaml' 2026-05-11T14:30:14,717 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_management.yaml' 2026-05-11T14:30:14,718 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_marketing.yaml' 2026-05-11T14:30:14,719 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_medical_genetics.yaml' 2026-05-11T14:30:14,720 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_miscellaneous.yaml' 2026-05-11T14:30:14,721 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_disputes.yaml' 2026-05-11T14:30:14,722 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:14,723 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_nutrition.yaml' 2026-05-11T14:30:14,725 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_philosophy.yaml' 2026-05-11T14:30:14,726 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_prehistory.yaml' 2026-05-11T14:30:14,727 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_accounting.yaml' 2026-05-11T14:30:14,728 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_law.yaml' 2026-05-11T14:30:14,729 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_medicine.yaml' 2026-05-11T14:30:14,730 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_psychology.yaml' 2026-05-11T14:30:14,731 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_public_relations.yaml' 2026-05-11T14:30:14,732 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_security_studies.yaml' 2026-05-11T14:30:14,733 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_sociology.yaml' 2026-05-11T14:30:14,735 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:14,736 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_virology.yaml' 2026-05-11T14:30:14,737 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_world_religions.yaml' 2026-05-11T14:30:14,740 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama.yaml' 2026-05-11T14:30:14,741 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama_template_yaml' 2026-05-11T14:30:14,742 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_humanities.yaml' 2026-05-11T14:30:14,743 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_other.yaml' 2026-05-11T14:30:14,744 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_social_sciences.yaml' 2026-05-11T14:30:14,745 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_stem.yaml' 2026-05-11T14:30:14,747 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:14,748 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_anatomy.yaml' 2026-05-11T14:30:14,749 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_astronomy.yaml' 2026-05-11T14:30:14,750 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_business_ethics.yaml' 2026-05-11T14:30:14,751 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:14,753 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_biology.yaml' 2026-05-11T14:30:14,754 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_chemistry.yaml' 2026-05-11T14:30:14,755 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_computer_science.yaml' 2026-05-11T14:30:14,756 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_mathematics.yaml' 2026-05-11T14:30:14,757 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_medicine.yaml' 2026-05-11T14:30:14,759 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_physics.yaml' 2026-05-11T14:30:14,760 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_computer_security.yaml' 2026-05-11T14:30:14,761 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:14,762 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_econometrics.yaml' 2026-05-11T14:30:14,763 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:14,764 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:14,765 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_formal_logic.yaml' 2026-05-11T14:30:14,766 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_global_facts.yaml' 2026-05-11T14:30:14,767 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_biology.yaml' 2026-05-11T14:30:14,768 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:14,769 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:14,770 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:14,772 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_geography.yaml' 2026-05-11T14:30:14,773 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:14,774 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:14,775 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:14,776 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:14,777 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_physics.yaml' 2026-05-11T14:30:14,778 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:14,780 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:14,781 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:14,782 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:14,783 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_aging.yaml' 2026-05-11T14:30:14,784 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_sexuality.yaml' 2026-05-11T14:30:14,786 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_international_law.yaml' 2026-05-11T14:30:14,787 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_jurisprudence.yaml' 2026-05-11T14:30:14,788 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:14,789 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_machine_learning.yaml' 2026-05-11T14:30:14,790 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_management.yaml' 2026-05-11T14:30:14,792 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_marketing.yaml' 2026-05-11T14:30:14,793 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_medical_genetics.yaml' 2026-05-11T14:30:14,794 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_miscellaneous.yaml' 2026-05-11T14:30:14,796 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_disputes.yaml' 2026-05-11T14:30:14,798 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:14,800 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_nutrition.yaml' 2026-05-11T14:30:14,801 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_philosophy.yaml' 2026-05-11T14:30:14,802 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_prehistory.yaml' 2026-05-11T14:30:14,803 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_accounting.yaml' 2026-05-11T14:30:14,805 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_law.yaml' 2026-05-11T14:30:14,806 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_medicine.yaml' 2026-05-11T14:30:14,808 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_psychology.yaml' 2026-05-11T14:30:14,809 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_public_relations.yaml' 2026-05-11T14:30:14,810 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_security_studies.yaml' 2026-05-11T14:30:14,811 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_sociology.yaml' 2026-05-11T14:30:14,812 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:14,813 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_virology.yaml' 2026-05-11T14:30:14,814 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_world_religions.yaml' 2026-05-11T14:30:14,817 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_continuation_template_yaml' 2026-05-11T14:30:14,818 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_humanities.yaml' 2026-05-11T14:30:14,819 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_llama.yaml' 2026-05-11T14:30:14,821 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_other.yaml' 2026-05-11T14:30:14,822 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_social_sciences.yaml' 2026-05-11T14:30:14,823 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_stem.yaml' 2026-05-11T14:30:14,824 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_abstract_algebra.yaml' 2026-05-11T14:30:14,826 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_anatomy.yaml' 2026-05-11T14:30:14,827 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_astronomy.yaml' 2026-05-11T14:30:14,828 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_business_ethics.yaml' 2026-05-11T14:30:14,830 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_clinical_knowledge.yaml' 2026-05-11T14:30:14,831 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_biology.yaml' 2026-05-11T14:30:14,832 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_chemistry.yaml' 2026-05-11T14:30:14,834 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_computer_science.yaml' 2026-05-11T14:30:14,835 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_mathematics.yaml' 2026-05-11T14:30:14,837 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_medicine.yaml' 2026-05-11T14:30:14,838 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_physics.yaml' 2026-05-11T14:30:14,840 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_computer_security.yaml' 2026-05-11T14:30:14,841 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_conceptual_physics.yaml' 2026-05-11T14:30:14,842 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_econometrics.yaml' 2026-05-11T14:30:14,844 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_electrical_engineering.yaml' 2026-05-11T14:30:14,845 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_elementary_mathematics.yaml' 2026-05-11T14:30:14,847 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_formal_logic.yaml' 2026-05-11T14:30:14,848 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_global_facts.yaml' 2026-05-11T14:30:14,850 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_biology.yaml' 2026-05-11T14:30:14,851 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_chemistry.yaml' 2026-05-11T14:30:14,853 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_computer_science.yaml' 2026-05-11T14:30:14,856 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_european_history.yaml' 2026-05-11T14:30:14,857 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_geography.yaml' 2026-05-11T14:30:14,859 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_government_and_politics.yaml' 2026-05-11T14:30:14,860 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_macroeconomics.yaml' 2026-05-11T14:30:14,862 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_mathematics.yaml' 2026-05-11T14:30:14,863 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_microeconomics.yaml' 2026-05-11T14:30:14,865 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_physics.yaml' 2026-05-11T14:30:14,866 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_psychology.yaml' 2026-05-11T14:30:14,867 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_statistics.yaml' 2026-05-11T14:30:14,870 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_us_history.yaml' 2026-05-11T14:30:14,871 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_world_history.yaml' 2026-05-11T14:30:14,873 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_aging.yaml' 2026-05-11T14:30:14,874 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_sexuality.yaml' 2026-05-11T14:30:14,875 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_international_law.yaml' 2026-05-11T14:30:14,877 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_jurisprudence.yaml' 2026-05-11T14:30:14,878 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_logical_fallacies.yaml' 2026-05-11T14:30:14,880 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_machine_learning.yaml' 2026-05-11T14:30:14,881 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_management.yaml' 2026-05-11T14:30:14,882 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_marketing.yaml' 2026-05-11T14:30:14,883 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_medical_genetics.yaml' 2026-05-11T14:30:14,885 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_miscellaneous.yaml' 2026-05-11T14:30:14,886 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_disputes.yaml' 2026-05-11T14:30:14,887 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_scenarios.yaml' 2026-05-11T14:30:14,889 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_nutrition.yaml' 2026-05-11T14:30:14,890 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_philosophy.yaml' 2026-05-11T14:30:14,892 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_prehistory.yaml' 2026-05-11T14:30:14,893 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_accounting.yaml' 2026-05-11T14:30:14,895 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_law.yaml' 2026-05-11T14:30:14,897 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_medicine.yaml' 2026-05-11T14:30:14,898 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_psychology.yaml' 2026-05-11T14:30:14,900 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_public_relations.yaml' 2026-05-11T14:30:14,901 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_security_studies.yaml' 2026-05-11T14:30:14,903 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_sociology.yaml' 2026-05-11T14:30:14,904 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_us_foreign_policy.yaml' 2026-05-11T14:30:14,906 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_virology.yaml' 2026-05-11T14:30:14,907 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_world_religions.yaml' 2026-05-11T14:30:14,908 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/utils.py' 2026-05-11T14:30:14,911 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_continuation_template_yaml' 2026-05-11T14:30:14,912 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_humanities.yaml' 2026-05-11T14:30:14,913 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_llama.yaml' 2026-05-11T14:30:14,915 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_other.yaml' 2026-05-11T14:30:14,916 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_social_sciences.yaml' 2026-05-11T14:30:14,917 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_stem.yaml' 2026-05-11T14:30:14,918 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_abstract_algebra.yaml' 2026-05-11T14:30:14,920 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_anatomy.yaml' 2026-05-11T14:30:14,921 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_astronomy.yaml' 2026-05-11T14:30:14,923 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_business_ethics.yaml' 2026-05-11T14:30:14,924 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_clinical_knowledge.yaml' 2026-05-11T14:30:14,925 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_biology.yaml' 2026-05-11T14:30:14,926 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_chemistry.yaml' 2026-05-11T14:30:14,928 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_computer_science.yaml' 2026-05-11T14:30:14,929 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_mathematics.yaml' 2026-05-11T14:30:14,930 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_medicine.yaml' 2026-05-11T14:30:14,932 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_physics.yaml' 2026-05-11T14:30:14,933 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_computer_security.yaml' 2026-05-11T14:30:14,935 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_conceptual_physics.yaml' 2026-05-11T14:30:14,936 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_econometrics.yaml' 2026-05-11T14:30:14,937 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_electrical_engineering.yaml' 2026-05-11T14:30:14,939 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_elementary_mathematics.yaml' 2026-05-11T14:30:14,940 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_formal_logic.yaml' 2026-05-11T14:30:14,941 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_global_facts.yaml' 2026-05-11T14:30:14,942 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_biology.yaml' 2026-05-11T14:30:14,944 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_chemistry.yaml' 2026-05-11T14:30:14,945 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_computer_science.yaml' 2026-05-11T14:30:14,948 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_european_history.yaml' 2026-05-11T14:30:14,949 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_geography.yaml' 2026-05-11T14:30:14,951 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_government_and_politics.yaml' 2026-05-11T14:30:14,952 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_macroeconomics.yaml' 2026-05-11T14:30:14,953 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_mathematics.yaml' 2026-05-11T14:30:14,955 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_microeconomics.yaml' 2026-05-11T14:30:14,957 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_physics.yaml' 2026-05-11T14:30:14,959 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_psychology.yaml' 2026-05-11T14:30:14,960 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_statistics.yaml' 2026-05-11T14:30:14,962 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml' 2026-05-11T14:30:14,964 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml' 2026-05-11T14:30:14,966 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml' 2026-05-11T14:30:14,967 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml' 2026-05-11T14:30:14,968 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml' 2026-05-11T14:30:14,970 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml' 2026-05-11T14:30:14,971 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml' 2026-05-11T14:30:14,972 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml' 2026-05-11T14:30:14,974 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml' 2026-05-11T14:30:14,975 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml' 2026-05-11T14:30:14,976 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml' 2026-05-11T14:30:14,978 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml' 2026-05-11T14:30:14,979 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml' 2026-05-11T14:30:14,980 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml' 2026-05-11T14:30:14,981 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml' 2026-05-11T14:30:14,983 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml' 2026-05-11T14:30:14,984 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml' 2026-05-11T14:30:14,985 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml' 2026-05-11T14:30:14,987 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml' 2026-05-11T14:30:14,989 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml' 2026-05-11T14:30:14,990 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_psychology.yaml' 2026-05-11T14:30:14,992 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_public_relations.yaml' 2026-05-11T14:30:14,993 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_security_studies.yaml' 2026-05-11T14:30:14,995 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_sociology.yaml' 2026-05-11T14:30:14,996 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_us_foreign_policy.yaml' 2026-05-11T14:30:14,998 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_virology.yaml' 2026-05-11T14:30:14,999 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_world_religions.yaml' 2026-05-11T14:30:15,001 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/utils.py' 2026-05-11T14:30:15,004 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_continuation_template_yaml' 2026-05-11T14:30:15,006 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_humanities.yaml' 2026-05-11T14:30:15,007 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_llama.yaml' 2026-05-11T14:30:15,008 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_other.yaml' 2026-05-11T14:30:15,009 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_social_sciences.yaml' 2026-05-11T14:30:15,010 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_stem.yaml' 2026-05-11T14:30:15,012 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_abstract_algebra.yaml' 2026-05-11T14:30:15,013 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_anatomy.yaml' 2026-05-11T14:30:15,014 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_astronomy.yaml' 2026-05-11T14:30:15,016 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_business_ethics.yaml' 2026-05-11T14:30:15,017 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_clinical_knowledge.yaml' 2026-05-11T14:30:15,018 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_biology.yaml' 2026-05-11T14:30:15,020 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml' 2026-05-11T14:30:15,021 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml' 2026-05-11T14:30:15,023 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml' 2026-05-11T14:30:15,024 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml' 2026-05-11T14:30:15,025 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml' 2026-05-11T14:30:15,027 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml' 2026-05-11T14:30:15,028 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml' 2026-05-11T14:30:15,030 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml' 2026-05-11T14:30:15,031 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml' 2026-05-11T14:30:15,032 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml' 2026-05-11T14:30:15,034 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml' 2026-05-11T14:30:15,035 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml' 2026-05-11T14:30:15,036 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml' 2026-05-11T14:30:15,038 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml' 2026-05-11T14:30:15,039 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml' 2026-05-11T14:30:15,042 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml' 2026-05-11T14:30:15,043 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml' 2026-05-11T14:30:15,044 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml' 2026-05-11T14:30:15,046 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml' 2026-05-11T14:30:15,047 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml' 2026-05-11T14:30:15,048 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_microeconomics.yaml' 2026-05-11T14:30:15,050 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_physics.yaml' 2026-05-11T14:30:15,051 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_psychology.yaml' 2026-05-11T14:30:15,052 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_statistics.yaml' 2026-05-11T14:30:15,055 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_us_history.yaml' 2026-05-11T14:30:15,057 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_world_history.yaml' 2026-05-11T14:30:15,058 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_aging.yaml' 2026-05-11T14:30:15,059 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_sexuality.yaml' 2026-05-11T14:30:15,061 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_international_law.yaml' 2026-05-11T14:30:15,062 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_jurisprudence.yaml' 2026-05-11T14:30:15,063 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_logical_fallacies.yaml' 2026-05-11T14:30:15,065 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_machine_learning.yaml' 2026-05-11T14:30:15,066 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_management.yaml' 2026-05-11T14:30:15,067 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_marketing.yaml' 2026-05-11T14:30:15,069 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_medical_genetics.yaml' 2026-05-11T14:30:15,070 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_miscellaneous.yaml' 2026-05-11T14:30:15,071 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_disputes.yaml' 2026-05-11T14:30:15,072 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_scenarios.yaml' 2026-05-11T14:30:15,074 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_nutrition.yaml' 2026-05-11T14:30:15,075 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_philosophy.yaml' 2026-05-11T14:30:15,077 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_prehistory.yaml' 2026-05-11T14:30:15,078 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_accounting.yaml' 2026-05-11T14:30:15,080 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_law.yaml' 2026-05-11T14:30:15,082 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_medicine.yaml' 2026-05-11T14:30:15,083 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_psychology.yaml' 2026-05-11T14:30:15,085 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_public_relations.yaml' 2026-05-11T14:30:15,087 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_security_studies.yaml' 2026-05-11T14:30:15,088 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_sociology.yaml' 2026-05-11T14:30:15,089 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_us_foreign_policy.yaml' 2026-05-11T14:30:15,091 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_virology.yaml' 2026-05-11T14:30:15,092 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_world_religions.yaml' 2026-05-11T14:30:15,094 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/utils.py' 2026-05-11T14:30:15,096 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_continuation_template_yaml' 2026-05-11T14:30:15,097 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_humanities.yaml' 2026-05-11T14:30:15,098 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_llama.yaml' 2026-05-11T14:30:15,099 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_other.yaml' 2026-05-11T14:30:15,100 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_social_sciences.yaml' 2026-05-11T14:30:15,101 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_stem.yaml' 2026-05-11T14:30:15,103 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_abstract_algebra.yaml' 2026-05-11T14:30:15,104 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_anatomy.yaml' 2026-05-11T14:30:15,106 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_astronomy.yaml' 2026-05-11T14:30:15,108 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_business_ethics.yaml' 2026-05-11T14:30:15,110 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_clinical_knowledge.yaml' 2026-05-11T14:30:15,112 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_biology.yaml' 2026-05-11T14:30:15,114 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_chemistry.yaml' 2026-05-11T14:30:15,117 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_computer_science.yaml' 2026-05-11T14:30:15,118 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_mathematics.yaml' 2026-05-11T14:30:15,120 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_medicine.yaml' 2026-05-11T14:30:15,122 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_physics.yaml' 2026-05-11T14:30:15,125 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_computer_security.yaml' 2026-05-11T14:30:15,127 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_conceptual_physics.yaml' 2026-05-11T14:30:15,130 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_econometrics.yaml' 2026-05-11T14:30:15,131 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_electrical_engineering.yaml' 2026-05-11T14:30:15,133 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_elementary_mathematics.yaml' 2026-05-11T14:30:15,135 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_formal_logic.yaml' 2026-05-11T14:30:15,137 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_global_facts.yaml' 2026-05-11T14:30:15,138 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_biology.yaml' 2026-05-11T14:30:15,140 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_chemistry.yaml' 2026-05-11T14:30:15,143 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_computer_science.yaml' 2026-05-11T14:30:15,152 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_european_history.yaml' 2026-05-11T14:30:15,154 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_geography.yaml' 2026-05-11T14:30:15,156 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_government_and_politics.yaml' 2026-05-11T14:30:15,158 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_macroeconomics.yaml' 2026-05-11T14:30:15,160 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_mathematics.yaml' 2026-05-11T14:30:15,162 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_microeconomics.yaml' 2026-05-11T14:30:15,164 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_physics.yaml' 2026-05-11T14:30:15,166 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_psychology.yaml' 2026-05-11T14:30:15,168 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_statistics.yaml' 2026-05-11T14:30:15,175 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_us_history.yaml' 2026-05-11T14:30:15,180 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_world_history.yaml' 2026-05-11T14:30:15,181 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_aging.yaml' 2026-05-11T14:30:15,183 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_sexuality.yaml' 2026-05-11T14:30:15,185 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_international_law.yaml' 2026-05-11T14:30:15,187 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_jurisprudence.yaml' 2026-05-11T14:30:15,189 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_logical_fallacies.yaml' 2026-05-11T14:30:15,191 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_machine_learning.yaml' 2026-05-11T14:30:15,193 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_management.yaml' 2026-05-11T14:30:15,194 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_marketing.yaml' 2026-05-11T14:30:15,196 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_medical_genetics.yaml' 2026-05-11T14:30:15,198 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_miscellaneous.yaml' 2026-05-11T14:30:15,200 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_disputes.yaml' 2026-05-11T14:30:15,201 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_scenarios.yaml' 2026-05-11T14:30:15,204 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_nutrition.yaml' 2026-05-11T14:30:15,205 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_philosophy.yaml' 2026-05-11T14:30:15,207 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_prehistory.yaml' 2026-05-11T14:30:15,209 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_accounting.yaml' 2026-05-11T14:30:15,214 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_law.yaml' 2026-05-11T14:30:15,218 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_medicine.yaml' 2026-05-11T14:30:15,220 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_psychology.yaml' 2026-05-11T14:30:15,222 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_public_relations.yaml' 2026-05-11T14:30:15,227 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_security_studies.yaml' 2026-05-11T14:30:15,229 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_sociology.yaml' 2026-05-11T14:30:15,231 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_us_foreign_policy.yaml' 2026-05-11T14:30:15,233 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_virology.yaml' 2026-05-11T14:30:15,234 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_world_religions.yaml' 2026-05-11T14:30:15,235 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/utils.py' 2026-05-11T14:30:15,238 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_continuation_template_yaml' 2026-05-11T14:30:15,239 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_humanities.yaml' 2026-05-11T14:30:15,241 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_llama.yaml' 2026-05-11T14:30:15,242 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_other.yaml' 2026-05-11T14:30:15,243 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_social_sciences.yaml' 2026-05-11T14:30:15,244 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_stem.yaml' 2026-05-11T14:30:15,245 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_abstract_algebra.yaml' 2026-05-11T14:30:15,247 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_anatomy.yaml' 2026-05-11T14:30:15,248 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_astronomy.yaml' 2026-05-11T14:30:15,249 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_business_ethics.yaml' 2026-05-11T14:30:15,251 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_clinical_knowledge.yaml' 2026-05-11T14:30:15,252 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_biology.yaml' 2026-05-11T14:30:15,253 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_chemistry.yaml' 2026-05-11T14:30:15,255 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_computer_science.yaml' 2026-05-11T14:30:15,256 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_mathematics.yaml' 2026-05-11T14:30:15,258 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_medicine.yaml' 2026-05-11T14:30:15,259 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_physics.yaml' 2026-05-11T14:30:15,260 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_computer_security.yaml' 2026-05-11T14:30:15,261 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_conceptual_physics.yaml' 2026-05-11T14:30:15,263 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_econometrics.yaml' 2026-05-11T14:30:15,264 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_electrical_engineering.yaml' 2026-05-11T14:30:15,265 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_elementary_mathematics.yaml' 2026-05-11T14:30:15,267 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_formal_logic.yaml' 2026-05-11T14:30:15,268 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_global_facts.yaml' 2026-05-11T14:30:15,269 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_biology.yaml' 2026-05-11T14:30:15,271 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_chemistry.yaml' 2026-05-11T14:30:15,272 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_computer_science.yaml' 2026-05-11T14:30:15,275 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_european_history.yaml' 2026-05-11T14:30:15,276 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_geography.yaml' 2026-05-11T14:30:15,278 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_government_and_politics.yaml' 2026-05-11T14:30:15,279 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_macroeconomics.yaml' 2026-05-11T14:30:15,280 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_mathematics.yaml' 2026-05-11T14:30:15,282 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_microeconomics.yaml' 2026-05-11T14:30:15,283 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_physics.yaml' 2026-05-11T14:30:15,284 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_psychology.yaml' 2026-05-11T14:30:15,286 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_statistics.yaml' 2026-05-11T14:30:15,288 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_us_history.yaml' 2026-05-11T14:30:15,290 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_world_history.yaml' 2026-05-11T14:30:15,291 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_aging.yaml' 2026-05-11T14:30:15,293 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_sexuality.yaml' 2026-05-11T14:30:15,294 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_international_law.yaml' 2026-05-11T14:30:15,295 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_jurisprudence.yaml' 2026-05-11T14:30:15,297 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_logical_fallacies.yaml' 2026-05-11T14:30:15,298 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_machine_learning.yaml' 2026-05-11T14:30:15,300 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_management.yaml' 2026-05-11T14:30:15,301 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_marketing.yaml' 2026-05-11T14:30:15,302 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_medical_genetics.yaml' 2026-05-11T14:30:15,304 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_miscellaneous.yaml' 2026-05-11T14:30:15,305 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_disputes.yaml' 2026-05-11T14:30:15,306 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_scenarios.yaml' 2026-05-11T14:30:15,308 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_nutrition.yaml' 2026-05-11T14:30:15,309 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_philosophy.yaml' 2026-05-11T14:30:15,310 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_prehistory.yaml' 2026-05-11T14:30:15,312 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_accounting.yaml' 2026-05-11T14:30:15,314 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_law.yaml' 2026-05-11T14:30:15,315 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_medicine.yaml' 2026-05-11T14:30:15,317 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_psychology.yaml' 2026-05-11T14:30:15,318 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_public_relations.yaml' 2026-05-11T14:30:15,319 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_security_studies.yaml' 2026-05-11T14:30:15,321 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_sociology.yaml' 2026-05-11T14:30:15,322 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_us_foreign_policy.yaml' 2026-05-11T14:30:15,323 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_virology.yaml' 2026-05-11T14:30:15,324 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_world_religions.yaml' 2026-05-11T14:30:15,326 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/utils.py' 2026-05-11T14:30:15,328 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/_default_template_yaml' 2026-05-11T14:30:15,329 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/_mmlu_pro.yaml' 2026-05-11T14:30:15,330 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_biology.yaml' 2026-05-11T14:30:15,331 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_business.yaml' 2026-05-11T14:30:15,332 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_chemistry.yaml' 2026-05-11T14:30:15,333 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_computer_science.yaml' 2026-05-11T14:30:15,335 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_economics.yaml' 2026-05-11T14:30:15,336 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_engineering.yaml' 2026-05-11T14:30:15,337 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_health.yaml' 2026-05-11T14:30:15,338 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_history.yaml' 2026-05-11T14:30:15,339 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_law.yaml' 2026-05-11T14:30:15,340 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_math.yaml' 2026-05-11T14:30:15,342 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_other.yaml' 2026-05-11T14:30:15,343 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_philosophy.yaml' 2026-05-11T14:30:15,344 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_physics.yaml' 2026-05-11T14:30:15,345 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_psychology.yaml' 2026-05-11T14:30:15,346 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/utils.py' 2026-05-11T14:30:15,349 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_continuation_template_yaml' 2026-05-11T14:30:15,350 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_humanities.yaml' 2026-05-11T14:30:15,351 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_llama.yaml' 2026-05-11T14:30:15,352 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_other.yaml' 2026-05-11T14:30:15,354 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_social_sciences.yaml' 2026-05-11T14:30:15,355 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_stem.yaml' 2026-05-11T14:30:15,356 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_abstract_algebra.yaml' 2026-05-11T14:30:15,358 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_anatomy.yaml' 2026-05-11T14:30:15,359 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_astronomy.yaml' 2026-05-11T14:30:15,361 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_business_ethics.yaml' 2026-05-11T14:30:15,362 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_clinical_knowledge.yaml' 2026-05-11T14:30:15,364 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_biology.yaml' 2026-05-11T14:30:15,365 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_chemistry.yaml' 2026-05-11T14:30:15,367 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_computer_science.yaml' 2026-05-11T14:30:15,369 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_mathematics.yaml' 2026-05-11T14:30:15,370 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_medicine.yaml' 2026-05-11T14:30:15,372 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_physics.yaml' 2026-05-11T14:30:15,373 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_computer_security.yaml' 2026-05-11T14:30:15,374 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_conceptual_physics.yaml' 2026-05-11T14:30:15,376 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_econometrics.yaml' 2026-05-11T14:30:15,377 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_electrical_engineering.yaml' 2026-05-11T14:30:15,378 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_elementary_mathematics.yaml' 2026-05-11T14:30:15,380 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_formal_logic.yaml' 2026-05-11T14:30:15,381 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_global_facts.yaml' 2026-05-11T14:30:15,382 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_biology.yaml' 2026-05-11T14:30:15,384 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_chemistry.yaml' 2026-05-11T14:30:15,385 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_computer_science.yaml' 2026-05-11T14:30:15,388 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_european_history.yaml' 2026-05-11T14:30:15,389 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_geography.yaml' 2026-05-11T14:30:15,391 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_government_and_politics.yaml' 2026-05-11T14:30:15,392 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_macroeconomics.yaml' 2026-05-11T14:30:15,393 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_mathematics.yaml' 2026-05-11T14:30:15,395 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_microeconomics.yaml' 2026-05-11T14:30:15,396 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_physics.yaml' 2026-05-11T14:30:15,397 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_psychology.yaml' 2026-05-11T14:30:15,399 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_statistics.yaml' 2026-05-11T14:30:15,401 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_us_history.yaml' 2026-05-11T14:30:15,403 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_world_history.yaml' 2026-05-11T14:30:15,404 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_aging.yaml' 2026-05-11T14:30:15,406 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_sexuality.yaml' 2026-05-11T14:30:15,407 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_international_law.yaml' 2026-05-11T14:30:15,409 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_jurisprudence.yaml' 2026-05-11T14:30:15,410 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_logical_fallacies.yaml' 2026-05-11T14:30:15,411 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_machine_learning.yaml' 2026-05-11T14:30:15,413 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_management.yaml' 2026-05-11T14:30:15,414 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_marketing.yaml' 2026-05-11T14:30:15,416 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_medical_genetics.yaml' 2026-05-11T14:30:15,417 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_miscellaneous.yaml' 2026-05-11T14:30:15,419 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_disputes.yaml' 2026-05-11T14:30:15,420 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_scenarios.yaml' 2026-05-11T14:30:15,422 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_nutrition.yaml' 2026-05-11T14:30:15,423 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_philosophy.yaml' 2026-05-11T14:30:15,425 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_prehistory.yaml' 2026-05-11T14:30:15,427 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_accounting.yaml' 2026-05-11T14:30:15,429 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_law.yaml' 2026-05-11T14:30:15,430 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_medicine.yaml' 2026-05-11T14:30:15,431 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_psychology.yaml' 2026-05-11T14:30:15,433 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_public_relations.yaml' 2026-05-11T14:30:15,434 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_security_studies.yaml' 2026-05-11T14:30:15,436 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_sociology.yaml' 2026-05-11T14:30:15,437 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_us_foreign_policy.yaml' 2026-05-11T14:30:15,439 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_virology.yaml' 2026-05-11T14:30:15,440 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_world_religions.yaml' 2026-05-11T14:30:15,441 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/utils.py' 2026-05-11T14:30:15,444 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_continuation_template_yaml' 2026-05-11T14:30:15,445 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_humanities.yaml' 2026-05-11T14:30:15,446 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_llama.yaml' 2026-05-11T14:30:15,447 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_other.yaml' 2026-05-11T14:30:15,448 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_social_sciences.yaml' 2026-05-11T14:30:15,449 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_stem.yaml' 2026-05-11T14:30:15,451 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_abstract_algebra.yaml' 2026-05-11T14:30:15,452 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_anatomy.yaml' 2026-05-11T14:30:15,455 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_astronomy.yaml' 2026-05-11T14:30:15,457 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_business_ethics.yaml' 2026-05-11T14:30:15,459 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_clinical_knowledge.yaml' 2026-05-11T14:30:15,460 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_biology.yaml' 2026-05-11T14:30:15,462 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_chemistry.yaml' 2026-05-11T14:30:15,464 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_computer_science.yaml' 2026-05-11T14:30:15,466 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_mathematics.yaml' 2026-05-11T14:30:15,468 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_medicine.yaml' 2026-05-11T14:30:15,470 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_physics.yaml' 2026-05-11T14:30:15,472 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_computer_security.yaml' 2026-05-11T14:30:15,474 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_conceptual_physics.yaml' 2026-05-11T14:30:15,476 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_econometrics.yaml' 2026-05-11T14:30:15,477 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_electrical_engineering.yaml' 2026-05-11T14:30:15,479 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_elementary_mathematics.yaml' 2026-05-11T14:30:15,481 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_formal_logic.yaml' 2026-05-11T14:30:15,483 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_global_facts.yaml' 2026-05-11T14:30:15,485 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_biology.yaml' 2026-05-11T14:30:15,487 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_chemistry.yaml' 2026-05-11T14:30:15,489 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_computer_science.yaml' 2026-05-11T14:30:15,499 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_european_history.yaml' 2026-05-11T14:30:15,501 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_geography.yaml' 2026-05-11T14:30:15,504 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_government_and_politics.yaml' 2026-05-11T14:30:15,506 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_macroeconomics.yaml' 2026-05-11T14:30:15,507 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_mathematics.yaml' 2026-05-11T14:30:15,509 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_microeconomics.yaml' 2026-05-11T14:30:15,511 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_physics.yaml' 2026-05-11T14:30:15,513 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_psychology.yaml' 2026-05-11T14:30:15,515 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_statistics.yaml' 2026-05-11T14:30:15,523 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_us_history.yaml' 2026-05-11T14:30:15,527 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_world_history.yaml' 2026-05-11T14:30:15,529 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_aging.yaml' 2026-05-11T14:30:15,531 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_sexuality.yaml' 2026-05-11T14:30:15,533 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_international_law.yaml' 2026-05-11T14:30:15,535 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_jurisprudence.yaml' 2026-05-11T14:30:15,537 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_logical_fallacies.yaml' 2026-05-11T14:30:15,539 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_machine_learning.yaml' 2026-05-11T14:30:15,541 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_management.yaml' 2026-05-11T14:30:15,543 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_marketing.yaml' 2026-05-11T14:30:15,544 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_medical_genetics.yaml' 2026-05-11T14:30:15,546 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_miscellaneous.yaml' 2026-05-11T14:30:15,548 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_disputes.yaml' 2026-05-11T14:30:15,550 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_scenarios.yaml' 2026-05-11T14:30:15,552 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_nutrition.yaml' 2026-05-11T14:30:15,554 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_philosophy.yaml' 2026-05-11T14:30:15,556 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_prehistory.yaml' 2026-05-11T14:30:15,558 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_accounting.yaml' 2026-05-11T14:30:15,563 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_law.yaml' 2026-05-11T14:30:15,567 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_medicine.yaml' 2026-05-11T14:30:15,569 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml' 2026-05-11T14:30:15,571 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml' 2026-05-11T14:30:15,575 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml' 2026-05-11T14:30:15,577 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml' 2026-05-11T14:30:15,579 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml' 2026-05-11T14:30:15,581 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml' 2026-05-11T14:30:15,583 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml' 2026-05-11T14:30:15,584 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/utils.py' 2026-05-11T14:30:15,588 adding 'lm_eval/tasks/lm_syneval/README.md' 2026-05-11T14:30:15,589 adding 'lm_eval/tasks/lm_syneval/_template_yaml' 2026-05-11T14:30:15,590 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__plur_MS_LMV_LMV.yaml' 2026-05-11T14:30:15,592 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__sing_MS_LMV_LMV.yaml' 2026-05-11T14:30:15,593 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_plur_ES_EV.yaml' 2026-05-11T14:30:15,594 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_sing_ES_EV.yaml' 2026-05-11T14:30:15,595 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_plur_ES_EV.yaml' 2026-05-11T14:30:15,596 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_sing_ES_EV.yaml' 2026-05-11T14:30:15,597 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_plur_ES_EV.yaml' 2026-05-11T14:30:15,598 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_sing_ES_EV.yaml' 2026-05-11T14:30:15,599 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_plur_ES_EV.yaml' 2026-05-11T14:30:15,600 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_sing_ES_EV.yaml' 2026-05-11T14:30:15,602 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_plur_ES_EV.yaml' 2026-05-11T14:30:15,603 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_sing_ES_EV.yaml' 2026-05-11T14:30:15,604 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_plur_ES_EV.yaml' 2026-05-11T14:30:15,605 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_sing_ES_EV.yaml' 2026-05-11T14:30:15,606 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_plur_ES_EV.yaml' 2026-05-11T14:30:15,607 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_sing_ES_EV.yaml' 2026-05-11T14:30:15,608 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_plur_ES_EV.yaml' 2026-05-11T14:30:15,609 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_sing_ES_EV.yaml' 2026-05-11T14:30:15,610 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_plur_MS_MV.yaml' 2026-05-11T14:30:15,611 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_sing_MS_MV.yaml' 2026-05-11T14:30:15,613 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_plur_MS_MV.yaml' 2026-05-11T14:30:15,614 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_sing_MS_MV.yaml' 2026-05-11T14:30:15,615 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_plur_IS_IV.yaml' 2026-05-11T14:30:15,616 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_sing_IS_IV.yaml' 2026-05-11T14:30:15,617 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_plur_IS_IV.yaml' 2026-05-11T14:30:15,618 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_sing_IS_IV.yaml' 2026-05-11T14:30:15,620 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_plur_MS_MV.yaml' 2026-05-11T14:30:15,621 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_sing_MS_MV.yaml' 2026-05-11T14:30:15,622 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_plur_MS_MV.yaml' 2026-05-11T14:30:15,623 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_sing_MS_MV.yaml' 2026-05-11T14:30:15,624 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_plur_IS_IV.yaml' 2026-05-11T14:30:15,625 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_sing_IS_IV.yaml' 2026-05-11T14:30:15,627 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_plur_IS_IV.yaml' 2026-05-11T14:30:15,628 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_sing_IS_IV.yaml' 2026-05-11T14:30:15,629 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_plur_ES.yaml' 2026-05-11T14:30:15,630 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_sing_ES.yaml' 2026-05-11T14:30:15,631 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_plur_ES.yaml' 2026-05-11T14:30:15,632 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_sing_ES.yaml' 2026-05-11T14:30:15,633 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_plur_ES.yaml' 2026-05-11T14:30:15,634 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_sing_ES.yaml' 2026-05-11T14:30:15,635 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_plur_ES.yaml' 2026-05-11T14:30:15,636 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_sing_ES.yaml' 2026-05-11T14:30:15,637 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_plur_BS.yaml' 2026-05-11T14:30:15,638 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_sing_BS.yaml' 2026-05-11T14:30:15,639 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_plur_BS.yaml' 2026-05-11T14:30:15,640 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_sing_BS.yaml' 2026-05-11T14:30:15,642 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__plur_MS_MV.yaml' 2026-05-11T14:30:15,643 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__sing_MS_MV.yaml' 2026-05-11T14:30:15,644 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_plur_ES.yaml' 2026-05-11T14:30:15,645 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_sing_ES.yaml' 2026-05-11T14:30:15,646 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_plur_ES.yaml' 2026-05-11T14:30:15,647 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_sing_ES.yaml' 2026-05-11T14:30:15,648 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__plur_MS_MV_MV.yaml' 2026-05-11T14:30:15,650 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__sing_MS_MV_MV.yaml' 2026-05-11T14:30:15,651 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__future.yaml' 2026-05-11T14:30:15,652 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__past.yaml' 2026-05-11T14:30:15,653 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__future.yaml' 2026-05-11T14:30:15,654 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__past.yaml' 2026-05-11T14:30:15,655 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__future.yaml' 2026-05-11T14:30:15,656 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__past.yaml' 2026-05-11T14:30:15,658 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__future.yaml' 2026-05-11T14:30:15,659 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__past.yaml' 2026-05-11T14:30:15,660 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_plur_BS.yaml' 2026-05-11T14:30:15,661 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_sing_BS.yaml' 2026-05-11T14:30:15,662 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_plur_BS.yaml' 2026-05-11T14:30:15,663 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_sing_BS.yaml' 2026-05-11T14:30:15,665 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_plur_ES_EV.yaml' 2026-05-11T14:30:15,666 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_sing_ES_EV.yaml' 2026-05-11T14:30:15,667 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_plur_ES_EV.yaml' 2026-05-11T14:30:15,668 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_sing_ES_EV.yaml' 2026-05-11T14:30:15,669 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__plur_MS_ANPHR.yaml' 2026-05-11T14:30:15,670 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__sing_MS_ANPHR.yaml' 2026-05-11T14:30:15,672 adding 'lm_eval/tasks/lm_syneval/lm_syneval_group.yaml' 2026-05-11T14:30:15,673 adding 'lm_eval/tasks/logiqa/README.md' 2026-05-11T14:30:15,674 adding 'lm_eval/tasks/logiqa/logiqa.yaml' 2026-05-11T14:30:15,675 adding 'lm_eval/tasks/logiqa/utils_logiqa.py' 2026-05-11T14:30:15,677 adding 'lm_eval/tasks/logiqa2/README.md' 2026-05-11T14:30:15,678 adding 'lm_eval/tasks/logiqa2/logieval.yaml' 2026-05-11T14:30:15,680 adding 'lm_eval/tasks/logiqa2/logiqa2.yaml' 2026-05-11T14:30:15,681 adding 'lm_eval/tasks/logiqa2/utils_logiqa2.py' 2026-05-11T14:30:15,683 adding 'lm_eval/tasks/longbench/2wikimqa.yaml' 2026-05-11T14:30:15,684 adding 'lm_eval/tasks/longbench/2wikimqa_e.yaml' 2026-05-11T14:30:15,686 adding 'lm_eval/tasks/longbench/README.md' 2026-05-11T14:30:15,688 adding 'lm_eval/tasks/longbench/_generate_config.py' 2026-05-11T14:30:15,689 adding 'lm_eval/tasks/longbench/_longbench.yaml' 2026-05-11T14:30:15,690 adding 'lm_eval/tasks/longbench/_longbench_code.yaml' 2026-05-11T14:30:15,691 adding 'lm_eval/tasks/longbench/_longbench_code_e.yaml' 2026-05-11T14:30:15,693 adding 'lm_eval/tasks/longbench/_longbench_e.yaml' 2026-05-11T14:30:15,694 adding 'lm_eval/tasks/longbench/_longbench_fewshot.yaml' 2026-05-11T14:30:15,695 adding 'lm_eval/tasks/longbench/_longbench_fewshot_e.yaml' 2026-05-11T14:30:15,696 adding 'lm_eval/tasks/longbench/_longbench_multi.yaml' 2026-05-11T14:30:15,697 adding 'lm_eval/tasks/longbench/_longbench_multi_e.yaml' 2026-05-11T14:30:15,698 adding 'lm_eval/tasks/longbench/_longbench_single.yaml' 2026-05-11T14:30:15,699 adding 'lm_eval/tasks/longbench/_longbench_single_e.yaml' 2026-05-11T14:30:15,701 adding 'lm_eval/tasks/longbench/_longbench_summarization.yaml' 2026-05-11T14:30:15,702 adding 'lm_eval/tasks/longbench/_longbench_summarization_e.yaml' 2026-05-11T14:30:15,703 adding 'lm_eval/tasks/longbench/_longbench_synthetic.yaml' 2026-05-11T14:30:15,704 adding 'lm_eval/tasks/longbench/_longbench_synthetic_e.yaml' 2026-05-11T14:30:15,705 adding 'lm_eval/tasks/longbench/dureader.yaml' 2026-05-11T14:30:15,707 adding 'lm_eval/tasks/longbench/gov_report.yaml' 2026-05-11T14:30:15,708 adding 'lm_eval/tasks/longbench/gov_report_e.yaml' 2026-05-11T14:30:15,709 adding 'lm_eval/tasks/longbench/hotpotqa.yaml' 2026-05-11T14:30:15,710 adding 'lm_eval/tasks/longbench/hotpotqa_e.yaml' 2026-05-11T14:30:15,712 adding 'lm_eval/tasks/longbench/lcc.yaml' 2026-05-11T14:30:15,713 adding 'lm_eval/tasks/longbench/lcc_e.yaml' 2026-05-11T14:30:15,714 adding 'lm_eval/tasks/longbench/lsht.yaml' 2026-05-11T14:30:15,716 adding 'lm_eval/tasks/longbench/metrics.py' 2026-05-11T14:30:15,717 adding 'lm_eval/tasks/longbench/multi_news.yaml' 2026-05-11T14:30:15,718 adding 'lm_eval/tasks/longbench/multi_news_e.yaml' 2026-05-11T14:30:15,720 adding 'lm_eval/tasks/longbench/multifieldqa_en.yaml' 2026-05-11T14:30:15,721 adding 'lm_eval/tasks/longbench/multifieldqa_en_e.yaml' 2026-05-11T14:30:15,722 adding 'lm_eval/tasks/longbench/multifieldqa_zh.yaml' 2026-05-11T14:30:15,723 adding 'lm_eval/tasks/longbench/musique.yaml' 2026-05-11T14:30:15,724 adding 'lm_eval/tasks/longbench/narrativeqa.yaml' 2026-05-11T14:30:15,726 adding 'lm_eval/tasks/longbench/passage_count.yaml' 2026-05-11T14:30:15,727 adding 'lm_eval/tasks/longbench/passage_count_e.yaml' 2026-05-11T14:30:15,728 adding 'lm_eval/tasks/longbench/passage_retrieval_en.yaml' 2026-05-11T14:30:15,729 adding 'lm_eval/tasks/longbench/passage_retrieval_en_e.yaml' 2026-05-11T14:30:15,731 adding 'lm_eval/tasks/longbench/passage_retrieval_zh.yaml' 2026-05-11T14:30:15,732 adding 'lm_eval/tasks/longbench/qasper.yaml' 2026-05-11T14:30:15,733 adding 'lm_eval/tasks/longbench/qasper_e.yaml' 2026-05-11T14:30:15,734 adding 'lm_eval/tasks/longbench/qmsum.yaml' 2026-05-11T14:30:15,735 adding 'lm_eval/tasks/longbench/repobench-p.yaml' 2026-05-11T14:30:15,736 adding 'lm_eval/tasks/longbench/repobench-p_e.yaml' 2026-05-11T14:30:15,737 adding 'lm_eval/tasks/longbench/samsum.yaml' 2026-05-11T14:30:15,738 adding 'lm_eval/tasks/longbench/samsum_e.yaml' 2026-05-11T14:30:15,740 adding 'lm_eval/tasks/longbench/trec.yaml' 2026-05-11T14:30:15,741 adding 'lm_eval/tasks/longbench/trec_e.yaml' 2026-05-11T14:30:15,742 adding 'lm_eval/tasks/longbench/triviaqa.yaml' 2026-05-11T14:30:15,743 adding 'lm_eval/tasks/longbench/triviaqa_e.yaml' 2026-05-11T14:30:15,744 adding 'lm_eval/tasks/longbench/utils.py' 2026-05-11T14:30:15,745 adding 'lm_eval/tasks/longbench/vcsum.yaml' 2026-05-11T14:30:15,748 adding 'lm_eval/tasks/longbench2/README.md' 2026-05-11T14:30:15,749 adding 'lm_eval/tasks/longbench2/_longbench2.yaml' 2026-05-11T14:30:15,750 adding 'lm_eval/tasks/longbench2/_longbench2_history.yaml' 2026-05-11T14:30:15,752 adding 'lm_eval/tasks/longbench2/_longbench2_incontext.yaml' 2026-05-11T14:30:15,753 adding 'lm_eval/tasks/longbench2/_longbench2_multi.yaml' 2026-05-11T14:30:15,754 adding 'lm_eval/tasks/longbench2/_longbench2_single.yaml' 2026-05-11T14:30:15,755 adding 'lm_eval/tasks/longbench2/_longbench2_structured.yaml' 2026-05-11T14:30:15,756 adding 'lm_eval/tasks/longbench2/_longbench_common_yaml' 2026-05-11T14:30:15,757 adding 'lm_eval/tasks/longbench2/academic_multi_doc.yaml' 2026-05-11T14:30:15,758 adding 'lm_eval/tasks/longbench2/academic_single.yaml' 2026-05-11T14:30:15,759 adding 'lm_eval/tasks/longbench2/agent_history.yaml' 2026-05-11T14:30:15,761 adding 'lm_eval/tasks/longbench2/detective.yaml' 2026-05-11T14:30:15,762 adding 'lm_eval/tasks/longbench2/dialogue_history.yaml' 2026-05-11T14:30:15,763 adding 'lm_eval/tasks/longbench2/event_order.yaml' 2026-05-11T14:30:15,764 adding 'lm_eval/tasks/longbench2/fin_multi_doc.yaml' 2026-05-11T14:30:15,765 adding 'lm_eval/tasks/longbench2/fin_single_doc.yaml' 2026-05-11T14:30:15,766 adding 'lm_eval/tasks/longbench2/govt_multi_doc.yaml' 2026-05-11T14:30:15,767 adding 'lm_eval/tasks/longbench2/govt_single_doc.yaml' 2026-05-11T14:30:15,768 adding 'lm_eval/tasks/longbench2/graph.yaml' 2026-05-11T14:30:15,769 adding 'lm_eval/tasks/longbench2/legal_multi.yaml' 2026-05-11T14:30:15,771 adding 'lm_eval/tasks/longbench2/legal_single.yaml' 2026-05-11T14:30:15,772 adding 'lm_eval/tasks/longbench2/lit_single_doc.yaml' 2026-05-11T14:30:15,773 adding 'lm_eval/tasks/longbench2/longbench2_code.yaml' 2026-05-11T14:30:15,774 adding 'lm_eval/tasks/longbench2/many_shot.yaml' 2026-05-11T14:30:15,775 adding 'lm_eval/tasks/longbench2/news_multi.yaml' 2026-05-11T14:30:15,776 adding 'lm_eval/tasks/longbench2/table.yaml' 2026-05-11T14:30:15,777 adding 'lm_eval/tasks/longbench2/translate.yaml' 2026-05-11T14:30:15,779 adding 'lm_eval/tasks/longbench2/user_guide.yaml' 2026-05-11T14:30:15,781 adding 'lm_eval/tasks/mastermind/README.md' 2026-05-11T14:30:15,782 adding 'lm_eval/tasks/mastermind/mastermind_24_easy.yaml' 2026-05-11T14:30:15,783 adding 'lm_eval/tasks/mastermind/mastermind_24_hard.yaml' 2026-05-11T14:30:15,784 adding 'lm_eval/tasks/mastermind/mastermind_35_easy.yaml' 2026-05-11T14:30:15,785 adding 'lm_eval/tasks/mastermind/mastermind_35_hard.yaml' 2026-05-11T14:30:15,787 adding 'lm_eval/tasks/mastermind/mastermind_46_easy.yaml' 2026-05-11T14:30:15,788 adding 'lm_eval/tasks/mastermind/mastermind_46_hard.yaml' 2026-05-11T14:30:15,790 adding 'lm_eval/tasks/mathqa/README.md' 2026-05-11T14:30:15,791 adding 'lm_eval/tasks/mathqa/mathqa.yaml' 2026-05-11T14:30:15,792 adding 'lm_eval/tasks/mathqa/utils.py' 2026-05-11T14:30:15,794 adding 'lm_eval/tasks/mbpp/README.md' 2026-05-11T14:30:15,796 adding 'lm_eval/tasks/mbpp/mbpp.yaml' 2026-05-11T14:30:15,797 adding 'lm_eval/tasks/mbpp/mbpp_instruct.yaml' 2026-05-11T14:30:15,798 adding 'lm_eval/tasks/mbpp/mbpp_plus.yaml' 2026-05-11T14:30:15,800 adding 'lm_eval/tasks/mbpp/mbpp_plus_instruct.yaml' 2026-05-11T14:30:15,801 adding 'lm_eval/tasks/mbpp/utils.py' 2026-05-11T14:30:15,803 adding 'lm_eval/tasks/mc_taco/README.md' 2026-05-11T14:30:15,804 adding 'lm_eval/tasks/mc_taco/default.yaml' 2026-05-11T14:30:15,806 adding 'lm_eval/tasks/med_concepts_qa/README.md' 2026-05-11T14:30:15,808 adding 'lm_eval/tasks/med_concepts_qa/_default_template_yaml' 2026-05-11T14:30:15,809 adding 'lm_eval/tasks/med_concepts_qa/_generate_configs.py' 2026-05-11T14:30:15,810 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa.yaml' 2026-05-11T14:30:15,811 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_atc.yaml' 2026-05-11T14:30:15,812 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10cm.yaml' 2026-05-11T14:30:15,813 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10proc.yaml' 2026-05-11T14:30:15,814 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9cm.yaml' 2026-05-11T14:30:15,816 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9proc.yaml' 2026-05-11T14:30:15,817 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_easy.yaml' 2026-05-11T14:30:15,818 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_hard.yaml' 2026-05-11T14:30:15,819 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_medium.yaml' 2026-05-11T14:30:15,820 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_easy.yaml' 2026-05-11T14:30:15,821 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_hard.yaml' 2026-05-11T14:30:15,822 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_medium.yaml' 2026-05-11T14:30:15,823 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_easy.yaml' 2026-05-11T14:30:15,824 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_hard.yaml' 2026-05-11T14:30:15,826 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_medium.yaml' 2026-05-11T14:30:15,827 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_easy.yaml' 2026-05-11T14:30:15,828 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_hard.yaml' 2026-05-11T14:30:15,829 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_medium.yaml' 2026-05-11T14:30:15,830 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_easy.yaml' 2026-05-11T14:30:15,832 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_hard.yaml' 2026-05-11T14:30:15,833 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_medium.yaml' 2026-05-11T14:30:15,835 adding 'lm_eval/tasks/med_prescriptions/med_prescriptions_easy.yaml' 2026-05-11T14:30:15,836 adding 'lm_eval/tasks/med_prescriptions/med_prescriptions_hard.yaml' 2026-05-11T14:30:15,847 adding 'lm_eval/tasks/med_prescriptions/utils.py' 2026-05-11T14:30:15,850 adding 'lm_eval/tasks/med_text_classification/med_text_classification_easy.yaml' 2026-05-11T14:30:15,851 adding 'lm_eval/tasks/med_text_classification/med_text_classification_hard.yaml' 2026-05-11T14:30:15,852 adding 'lm_eval/tasks/med_text_classification/utils.py' 2026-05-11T14:30:15,855 adding 'lm_eval/tasks/meddialog/README.md' 2026-05-11T14:30:15,856 adding 'lm_eval/tasks/meddialog/meddialog_qsumm.yaml' 2026-05-11T14:30:15,857 adding 'lm_eval/tasks/meddialog/meddialog_qsumm_perplexity.yaml' 2026-05-11T14:30:15,859 adding 'lm_eval/tasks/meddialog/meddialog_raw_dialogues.yaml' 2026-05-11T14:30:15,860 adding 'lm_eval/tasks/meddialog/meddialog_raw_perplexity.yaml' 2026-05-11T14:30:15,862 adding 'lm_eval/tasks/meddialog/utils.py' 2026-05-11T14:30:15,863 adding 'lm_eval/tasks/meddialog/utils_perplexity.py' 2026-05-11T14:30:15,865 adding 'lm_eval/tasks/mediqa_qa2019/README.md' 2026-05-11T14:30:15,866 adding 'lm_eval/tasks/mediqa_qa2019/mediqa_qa2019.yaml' 2026-05-11T14:30:15,867 adding 'lm_eval/tasks/mediqa_qa2019/mediqa_qa2019_perplexity.yaml' 2026-05-11T14:30:15,868 adding 'lm_eval/tasks/mediqa_qa2019/utils.py' 2026-05-11T14:30:15,869 adding 'lm_eval/tasks/mediqa_qa2019/utils_perplexity.py' 2026-05-11T14:30:15,871 adding 'lm_eval/tasks/medmcqa/medmcqa.yaml' 2026-05-11T14:30:15,872 adding 'lm_eval/tasks/medmcqa/utils_medmcqa.py' 2026-05-11T14:30:15,874 adding 'lm_eval/tasks/medqa/medqa.yaml' 2026-05-11T14:30:15,875 adding 'lm_eval/tasks/medqa/preprocess_medqa.py' 2026-05-11T14:30:15,876 adding 'lm_eval/tasks/medtext/README.md' 2026-05-11T14:30:15,878 adding 'lm_eval/tasks/medtext/medtext.yaml' 2026-05-11T14:30:15,879 adding 'lm_eval/tasks/medtext/medtext_perplexity.yaml' 2026-05-11T14:30:15,880 adding 'lm_eval/tasks/medtext/utils.py' 2026-05-11T14:30:15,881 adding 'lm_eval/tasks/medtext/utils_perplexity.py' 2026-05-11T14:30:15,883 adding 'lm_eval/tasks/mela/README.md' 2026-05-11T14:30:15,884 adding 'lm_eval/tasks/mela/_mela.yaml' 2026-05-11T14:30:15,885 adding 'lm_eval/tasks/mela/mela_ar.yaml' 2026-05-11T14:30:15,886 adding 'lm_eval/tasks/mela/mela_de.yaml' 2026-05-11T14:30:15,887 adding 'lm_eval/tasks/mela/mela_en.yaml' 2026-05-11T14:30:15,889 adding 'lm_eval/tasks/mela/mela_es.yaml' 2026-05-11T14:30:15,890 adding 'lm_eval/tasks/mela/mela_fr.yaml' 2026-05-11T14:30:15,891 adding 'lm_eval/tasks/mela/mela_is.yaml' 2026-05-11T14:30:15,892 adding 'lm_eval/tasks/mela/mela_it.yaml' 2026-05-11T14:30:15,893 adding 'lm_eval/tasks/mela/mela_ja.yaml' 2026-05-11T14:30:15,894 adding 'lm_eval/tasks/mela/mela_ru.yaml' 2026-05-11T14:30:15,895 adding 'lm_eval/tasks/mela/mela_zh.yaml' 2026-05-11T14:30:15,897 adding 'lm_eval/tasks/meqsum/README.md' 2026-05-11T14:30:15,898 adding 'lm_eval/tasks/meqsum/meqsum.yaml' 2026-05-11T14:30:15,899 adding 'lm_eval/tasks/meqsum/utils.py' 2026-05-11T14:30:15,902 adding 'lm_eval/tasks/metabench/README.md' 2026-05-11T14:30:15,903 adding 'lm_eval/tasks/metabench/metabench.yaml' 2026-05-11T14:30:15,904 adding 'lm_eval/tasks/metabench/metabench_arc.yaml' 2026-05-11T14:30:15,905 adding 'lm_eval/tasks/metabench/metabench_arc_permute.yaml' 2026-05-11T14:30:15,906 adding 'lm_eval/tasks/metabench/metabench_arc_secondary.yaml' 2026-05-11T14:30:15,907 adding 'lm_eval/tasks/metabench/metabench_arc_secondary_permute.yaml' 2026-05-11T14:30:15,909 adding 'lm_eval/tasks/metabench/metabench_gsm8k.yaml' 2026-05-11T14:30:15,910 adding 'lm_eval/tasks/metabench/metabench_gsm8k_secondary.yaml' 2026-05-11T14:30:15,911 adding 'lm_eval/tasks/metabench/metabench_hellaswag.yaml' 2026-05-11T14:30:15,912 adding 'lm_eval/tasks/metabench/metabench_hellaswag_permute.yaml' 2026-05-11T14:30:15,913 adding 'lm_eval/tasks/metabench/metabench_hellaswag_secondary.yaml' 2026-05-11T14:30:15,914 adding 'lm_eval/tasks/metabench/metabench_hellaswag_secondary_permute.yaml' 2026-05-11T14:30:15,916 adding 'lm_eval/tasks/metabench/metabench_mmlu.yaml' 2026-05-11T14:30:15,917 adding 'lm_eval/tasks/metabench/metabench_mmlu_permute.yaml' 2026-05-11T14:30:15,918 adding 'lm_eval/tasks/metabench/metabench_mmlu_secondary.yaml' 2026-05-11T14:30:15,919 adding 'lm_eval/tasks/metabench/metabench_mmlu_secondary_permute.yaml' 2026-05-11T14:30:15,920 adding 'lm_eval/tasks/metabench/metabench_permute.yaml' 2026-05-11T14:30:15,921 adding 'lm_eval/tasks/metabench/metabench_secondary.yaml' 2026-05-11T14:30:15,923 adding 'lm_eval/tasks/metabench/metabench_secondary_permute.yaml' 2026-05-11T14:30:15,924 adding 'lm_eval/tasks/metabench/metabench_truthfulqa.yaml' 2026-05-11T14:30:15,925 adding 'lm_eval/tasks/metabench/metabench_truthfulqa_permute.yaml' 2026-05-11T14:30:15,926 adding 'lm_eval/tasks/metabench/metabench_truthfulqa_secondary.yaml' 2026-05-11T14:30:15,927 adding 'lm_eval/tasks/metabench/metabench_truthfulqa_secondary_permute.yaml' 2026-05-11T14:30:15,929 adding 'lm_eval/tasks/metabench/metabench_winogrande.yaml' 2026-05-11T14:30:15,930 adding 'lm_eval/tasks/metabench/metabench_winogrande_permute.yaml' 2026-05-11T14:30:15,931 adding 'lm_eval/tasks/metabench/metabench_winogrande_secondary.yaml' 2026-05-11T14:30:15,932 adding 'lm_eval/tasks/metabench/metabench_winogrande_secondary_permute.yaml' 2026-05-11T14:30:15,933 adding 'lm_eval/tasks/metabench/process_docs.py' 2026-05-11T14:30:15,935 adding 'lm_eval/tasks/metabench/process_docs_permute.py' 2026-05-11T14:30:15,937 adding 'lm_eval/tasks/mgsm/README.md' 2026-05-11T14:30:15,938 adding 'lm_eval/tasks/mgsm/gen_yaml.sh' 2026-05-11T14:30:15,940 adding 'lm_eval/tasks/mgsm/utils.py' 2026-05-11T14:30:15,941 adding 'lm_eval/tasks/mgsm/direct/direct_yaml' 2026-05-11T14:30:15,942 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml' 2026-05-11T14:30:15,943 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml' 2026-05-11T14:30:15,945 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml' 2026-05-11T14:30:15,946 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml' 2026-05-11T14:30:15,947 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml' 2026-05-11T14:30:15,948 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml' 2026-05-11T14:30:15,949 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml' 2026-05-11T14:30:15,950 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml' 2026-05-11T14:30:15,951 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml' 2026-05-11T14:30:15,953 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml' 2026-05-11T14:30:15,954 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml' 2026-05-11T14:30:15,956 adding 'lm_eval/tasks/mgsm/en_cot/cot_yaml' 2026-05-11T14:30:15,957 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml' 2026-05-11T14:30:15,958 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml' 2026-05-11T14:30:15,959 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml' 2026-05-11T14:30:15,960 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml' 2026-05-11T14:30:15,961 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml' 2026-05-11T14:30:15,963 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml' 2026-05-11T14:30:15,964 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml' 2026-05-11T14:30:15,965 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml' 2026-05-11T14:30:15,966 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml' 2026-05-11T14:30:15,967 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml' 2026-05-11T14:30:15,968 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml' 2026-05-11T14:30:15,970 adding 'lm_eval/tasks/mgsm/native_cot/cot_yaml' 2026-05-11T14:30:15,971 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml' 2026-05-11T14:30:15,973 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml' 2026-05-11T14:30:15,974 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml' 2026-05-11T14:30:15,975 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml' 2026-05-11T14:30:15,976 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml' 2026-05-11T14:30:15,978 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml' 2026-05-11T14:30:15,979 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml' 2026-05-11T14:30:15,980 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml' 2026-05-11T14:30:15,981 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml' 2026-05-11T14:30:15,983 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml' 2026-05-11T14:30:15,984 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml' 2026-05-11T14:30:15,986 adding 'lm_eval/tasks/mimic_repsum/README.md' 2026-05-11T14:30:15,987 adding 'lm_eval/tasks/mimic_repsum/mimic_repsum.yaml' 2026-05-11T14:30:15,988 adding 'lm_eval/tasks/mimic_repsum/mimic_repsum_perplexity.yaml' 2026-05-11T14:30:15,989 adding 'lm_eval/tasks/mimic_repsum/utils.py' 2026-05-11T14:30:15,991 adding 'lm_eval/tasks/mimic_repsum/utils_perplexity.py' 2026-05-11T14:30:15,993 adding 'lm_eval/tasks/minerva_math/README.md' 2026-05-11T14:30:15,994 adding 'lm_eval/tasks/minerva_math/minerva_math500.yaml' 2026-05-11T14:30:15,995 adding 'lm_eval/tasks/minerva_math/minerva_math_algebra.yaml' 2026-05-11T14:30:15,996 adding 'lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml' 2026-05-11T14:30:15,997 adding 'lm_eval/tasks/minerva_math/minerva_math_geometry.yaml' 2026-05-11T14:30:15,998 adding 'lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml' 2026-05-11T14:30:15,999 adding 'lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml' 2026-05-11T14:30:16,000 adding 'lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml' 2026-05-11T14:30:16,001 adding 'lm_eval/tasks/minerva_math/minerva_math_precalc.yaml' 2026-05-11T14:30:16,003 adding 'lm_eval/tasks/minerva_math/utils.py' 2026-05-11T14:30:16,006 adding 'lm_eval/tasks/mlqa/README.md' 2026-05-11T14:30:16,007 adding 'lm_eval/tasks/mlqa/generate_tasks.py' 2026-05-11T14:30:16,008 adding 'lm_eval/tasks/mlqa/mlqa_ar_ar.yaml' 2026-05-11T14:30:16,009 adding 'lm_eval/tasks/mlqa/mlqa_ar_de.yaml' 2026-05-11T14:30:16,010 adding 'lm_eval/tasks/mlqa/mlqa_ar_en.yaml' 2026-05-11T14:30:16,011 adding 'lm_eval/tasks/mlqa/mlqa_ar_es.yaml' 2026-05-11T14:30:16,012 adding 'lm_eval/tasks/mlqa/mlqa_ar_hi.yaml' 2026-05-11T14:30:16,013 adding 'lm_eval/tasks/mlqa/mlqa_ar_vi.yaml' 2026-05-11T14:30:16,015 adding 'lm_eval/tasks/mlqa/mlqa_ar_zh.yaml' 2026-05-11T14:30:16,016 adding 'lm_eval/tasks/mlqa/mlqa_common_yaml' 2026-05-11T14:30:16,017 adding 'lm_eval/tasks/mlqa/mlqa_de_ar.yaml' 2026-05-11T14:30:16,018 adding 'lm_eval/tasks/mlqa/mlqa_de_de.yaml' 2026-05-11T14:30:16,019 adding 'lm_eval/tasks/mlqa/mlqa_de_en.yaml' 2026-05-11T14:30:16,020 adding 'lm_eval/tasks/mlqa/mlqa_de_es.yaml' 2026-05-11T14:30:16,021 adding 'lm_eval/tasks/mlqa/mlqa_de_hi.yaml' 2026-05-11T14:30:16,022 adding 'lm_eval/tasks/mlqa/mlqa_de_vi.yaml' 2026-05-11T14:30:16,024 adding 'lm_eval/tasks/mlqa/mlqa_de_zh.yaml' 2026-05-11T14:30:16,025 adding 'lm_eval/tasks/mlqa/mlqa_en_ar.yaml' 2026-05-11T14:30:16,026 adding 'lm_eval/tasks/mlqa/mlqa_en_de.yaml' 2026-05-11T14:30:16,027 adding 'lm_eval/tasks/mlqa/mlqa_en_en.yaml' 2026-05-11T14:30:16,028 adding 'lm_eval/tasks/mlqa/mlqa_en_es.yaml' 2026-05-11T14:30:16,029 adding 'lm_eval/tasks/mlqa/mlqa_en_hi.yaml' 2026-05-11T14:30:16,030 adding 'lm_eval/tasks/mlqa/mlqa_en_vi.yaml' 2026-05-11T14:30:16,031 adding 'lm_eval/tasks/mlqa/mlqa_en_zh.yaml' 2026-05-11T14:30:16,032 adding 'lm_eval/tasks/mlqa/mlqa_es_ar.yaml' 2026-05-11T14:30:16,034 adding 'lm_eval/tasks/mlqa/mlqa_es_de.yaml' 2026-05-11T14:30:16,035 adding 'lm_eval/tasks/mlqa/mlqa_es_en.yaml' 2026-05-11T14:30:16,036 adding 'lm_eval/tasks/mlqa/mlqa_es_es.yaml' 2026-05-11T14:30:16,037 adding 'lm_eval/tasks/mlqa/mlqa_es_hi.yaml' 2026-05-11T14:30:16,038 adding 'lm_eval/tasks/mlqa/mlqa_es_vi.yaml' 2026-05-11T14:30:16,039 adding 'lm_eval/tasks/mlqa/mlqa_es_zh.yaml' 2026-05-11T14:30:16,040 adding 'lm_eval/tasks/mlqa/mlqa_hi_ar.yaml' 2026-05-11T14:30:16,042 adding 'lm_eval/tasks/mlqa/mlqa_hi_de.yaml' 2026-05-11T14:30:16,043 adding 'lm_eval/tasks/mlqa/mlqa_hi_en.yaml' 2026-05-11T14:30:16,044 adding 'lm_eval/tasks/mlqa/mlqa_hi_es.yaml' 2026-05-11T14:30:16,045 adding 'lm_eval/tasks/mlqa/mlqa_hi_hi.yaml' 2026-05-11T14:30:16,046 adding 'lm_eval/tasks/mlqa/mlqa_hi_vi.yaml' 2026-05-11T14:30:16,047 adding 'lm_eval/tasks/mlqa/mlqa_hi_zh.yaml' 2026-05-11T14:30:16,049 adding 'lm_eval/tasks/mlqa/mlqa_vi_ar.yaml' 2026-05-11T14:30:16,050 adding 'lm_eval/tasks/mlqa/mlqa_vi_de.yaml' 2026-05-11T14:30:16,051 adding 'lm_eval/tasks/mlqa/mlqa_vi_en.yaml' 2026-05-11T14:30:16,052 adding 'lm_eval/tasks/mlqa/mlqa_vi_es.yaml' 2026-05-11T14:30:16,053 adding 'lm_eval/tasks/mlqa/mlqa_vi_hi.yaml' 2026-05-11T14:30:16,054 adding 'lm_eval/tasks/mlqa/mlqa_vi_vi.yaml' 2026-05-11T14:30:16,055 adding 'lm_eval/tasks/mlqa/mlqa_vi_zh.yaml' 2026-05-11T14:30:16,057 adding 'lm_eval/tasks/mlqa/mlqa_zh_ar.yaml' 2026-05-11T14:30:16,058 adding 'lm_eval/tasks/mlqa/mlqa_zh_de.yaml' 2026-05-11T14:30:16,059 adding 'lm_eval/tasks/mlqa/mlqa_zh_en.yaml' 2026-05-11T14:30:16,060 adding 'lm_eval/tasks/mlqa/mlqa_zh_es.yaml' 2026-05-11T14:30:16,061 adding 'lm_eval/tasks/mlqa/mlqa_zh_hi.yaml' 2026-05-11T14:30:16,062 adding 'lm_eval/tasks/mlqa/mlqa_zh_vi.yaml' 2026-05-11T14:30:16,063 adding 'lm_eval/tasks/mlqa/mlqa_zh_zh.yaml' 2026-05-11T14:30:16,065 adding 'lm_eval/tasks/mlqa/utils.py' 2026-05-11T14:30:16,067 adding 'lm_eval/tasks/mmlu/README.md' 2026-05-11T14:30:16,068 adding 'lm_eval/tasks/mmlu/_generate_configs.py' 2026-05-11T14:30:16,070 adding 'lm_eval/tasks/mmlu/continuation/_continuation_template_yaml' 2026-05-11T14:30:16,072 adding 'lm_eval/tasks/mmlu/continuation/_mmlu.yaml' 2026-05-11T14:30:16,073 adding 'lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,074 adding 'lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml' 2026-05-11T14:30:16,075 adding 'lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml' 2026-05-11T14:30:16,076 adding 'lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,077 adding 'lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,078 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml' 2026-05-11T14:30:16,079 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,080 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,082 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,083 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,084 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml' 2026-05-11T14:30:16,085 adding 'lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml' 2026-05-11T14:30:16,086 adding 'lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,087 adding 'lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml' 2026-05-11T14:30:16,088 adding 'lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,089 adding 'lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,090 adding 'lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,092 adding 'lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml' 2026-05-11T14:30:16,094 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,097 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,098 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,099 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,100 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,102 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,103 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,104 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,105 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,106 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,107 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,109 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,110 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,111 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,112 adding 'lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml' 2026-05-11T14:30:16,113 adding 'lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,114 adding 'lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml' 2026-05-11T14:30:16,116 adding 'lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,117 adding 'lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,118 adding 'lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,119 adding 'lm_eval/tasks/mmlu/continuation/mmlu_management.yaml' 2026-05-11T14:30:16,120 adding 'lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml' 2026-05-11T14:30:16,121 adding 'lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,122 adding 'lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,123 adding 'lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,124 adding 'lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,126 adding 'lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml' 2026-05-11T14:30:16,127 adding 'lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml' 2026-05-11T14:30:16,128 adding 'lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml' 2026-05-11T14:30:16,129 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,130 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml' 2026-05-11T14:30:16,131 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,132 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,134 adding 'lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml' 2026-05-11T14:30:16,135 adding 'lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml' 2026-05-11T14:30:16,136 adding 'lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml' 2026-05-11T14:30:16,137 adding 'lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,138 adding 'lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml' 2026-05-11T14:30:16,139 adding 'lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml' 2026-05-11T14:30:16,142 adding 'lm_eval/tasks/mmlu/default/_default_template_yaml' 2026-05-11T14:30:16,143 adding 'lm_eval/tasks/mmlu/default/_mmlu.yaml' 2026-05-11T14:30:16,144 adding 'lm_eval/tasks/mmlu/default/_mmlu_humanities.yaml' 2026-05-11T14:30:16,145 adding 'lm_eval/tasks/mmlu/default/_mmlu_other.yaml' 2026-05-11T14:30:16,147 adding 'lm_eval/tasks/mmlu/default/_mmlu_social_sciences.yaml' 2026-05-11T14:30:16,148 adding 'lm_eval/tasks/mmlu/default/_mmlu_stem.yaml' 2026-05-11T14:30:16,149 adding 'lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,150 adding 'lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml' 2026-05-11T14:30:16,151 adding 'lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml' 2026-05-11T14:30:16,152 adding 'lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,153 adding 'lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,155 adding 'lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml' 2026-05-11T14:30:16,156 adding 'lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,157 adding 'lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,158 adding 'lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,159 adding 'lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,161 adding 'lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml' 2026-05-11T14:30:16,162 adding 'lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml' 2026-05-11T14:30:16,163 adding 'lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,164 adding 'lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml' 2026-05-11T14:30:16,165 adding 'lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,167 adding 'lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,168 adding 'lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,169 adding 'lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml' 2026-05-11T14:30:16,170 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,171 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,173 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,174 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,175 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,176 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,177 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,178 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,179 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,180 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,181 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,182 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,184 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,185 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,186 adding 'lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml' 2026-05-11T14:30:16,187 adding 'lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,188 adding 'lm_eval/tasks/mmlu/default/mmlu_international_law.yaml' 2026-05-11T14:30:16,189 adding 'lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,191 adding 'lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,192 adding 'lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,193 adding 'lm_eval/tasks/mmlu/default/mmlu_management.yaml' 2026-05-11T14:30:16,194 adding 'lm_eval/tasks/mmlu/default/mmlu_marketing.yaml' 2026-05-11T14:30:16,195 adding 'lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,196 adding 'lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,198 adding 'lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,199 adding 'lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,200 adding 'lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml' 2026-05-11T14:30:16,201 adding 'lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml' 2026-05-11T14:30:16,202 adding 'lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml' 2026-05-11T14:30:16,203 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,205 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml' 2026-05-11T14:30:16,206 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,207 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,208 adding 'lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml' 2026-05-11T14:30:16,209 adding 'lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml' 2026-05-11T14:30:16,211 adding 'lm_eval/tasks/mmlu/default/mmlu_sociology.yaml' 2026-05-11T14:30:16,212 adding 'lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,213 adding 'lm_eval/tasks/mmlu/default/mmlu_virology.yaml' 2026-05-11T14:30:16,214 adding 'lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml' 2026-05-11T14:30:16,254 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json' 2026-05-11T14:30:16,256 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml' 2026-05-11T14:30:16,258 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml' 2026-05-11T14:30:16,259 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,261 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml' 2026-05-11T14:30:16,262 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml' 2026-05-11T14:30:16,264 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,265 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,267 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml' 2026-05-11T14:30:16,268 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,270 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,271 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,273 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,274 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml' 2026-05-11T14:30:16,275 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml' 2026-05-11T14:30:16,277 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,279 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml' 2026-05-11T14:30:16,280 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,281 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,283 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,284 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml' 2026-05-11T14:30:16,286 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,287 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,289 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,292 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,293 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,295 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,296 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,298 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,299 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,300 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,302 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,303 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,306 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,307 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,309 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml' 2026-05-11T14:30:16,310 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,312 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml' 2026-05-11T14:30:16,314 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,315 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,317 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,319 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml' 2026-05-11T14:30:16,320 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml' 2026-05-11T14:30:16,322 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,324 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,325 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,327 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,329 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml' 2026-05-11T14:30:16,330 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml' 2026-05-11T14:30:16,331 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml' 2026-05-11T14:30:16,333 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,335 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml' 2026-05-11T14:30:16,336 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,338 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,339 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml' 2026-05-11T14:30:16,341 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml' 2026-05-11T14:30:16,342 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml' 2026-05-11T14:30:16,343 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,345 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml' 2026-05-11T14:30:16,346 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml' 2026-05-11T14:30:16,349 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml' 2026-05-11T14:30:16,350 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml' 2026-05-11T14:30:16,351 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,352 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml' 2026-05-11T14:30:16,353 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml' 2026-05-11T14:30:16,354 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,355 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,357 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml' 2026-05-11T14:30:16,358 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,359 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,360 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,361 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,362 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml' 2026-05-11T14:30:16,364 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml' 2026-05-11T14:30:16,365 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,366 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml' 2026-05-11T14:30:16,367 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,368 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,369 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,371 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml' 2026-05-11T14:30:16,372 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,373 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,374 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,375 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,377 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,378 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,379 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,380 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,381 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,383 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,384 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,385 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,386 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,387 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,388 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml' 2026-05-11T14:30:16,389 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,390 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml' 2026-05-11T14:30:16,391 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,392 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,393 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,394 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml' 2026-05-11T14:30:16,396 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml' 2026-05-11T14:30:16,397 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,398 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,399 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,401 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,402 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml' 2026-05-11T14:30:16,403 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml' 2026-05-11T14:30:16,405 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml' 2026-05-11T14:30:16,406 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,407 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml' 2026-05-11T14:30:16,408 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,409 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,411 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml' 2026-05-11T14:30:16,412 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml' 2026-05-11T14:30:16,413 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml' 2026-05-11T14:30:16,414 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,415 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml' 2026-05-11T14:30:16,417 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml' 2026-05-11T14:30:16,418 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py' 2026-05-11T14:30:16,421 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml' 2026-05-11T14:30:16,423 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml' 2026-05-11T14:30:16,424 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,425 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml' 2026-05-11T14:30:16,426 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml' 2026-05-11T14:30:16,428 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,429 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,430 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml' 2026-05-11T14:30:16,431 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,432 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,434 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,435 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,436 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml' 2026-05-11T14:30:16,437 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml' 2026-05-11T14:30:16,438 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,439 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml' 2026-05-11T14:30:16,440 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,441 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,442 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,444 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml' 2026-05-11T14:30:16,445 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,446 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,447 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,448 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,450 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,451 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,452 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,453 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,454 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,455 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,456 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,458 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,459 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,460 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,461 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml' 2026-05-11T14:30:16,463 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,464 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml' 2026-05-11T14:30:16,465 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,466 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,467 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,469 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml' 2026-05-11T14:30:16,470 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml' 2026-05-11T14:30:16,471 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,472 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,473 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,475 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,476 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml' 2026-05-11T14:30:16,477 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml' 2026-05-11T14:30:16,478 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml' 2026-05-11T14:30:16,480 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,481 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml' 2026-05-11T14:30:16,482 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,483 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,484 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml' 2026-05-11T14:30:16,485 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml' 2026-05-11T14:30:16,487 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml' 2026-05-11T14:30:16,488 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,489 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml' 2026-05-11T14:30:16,490 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml' 2026-05-11T14:30:16,491 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py' 2026-05-11T14:30:16,494 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml' 2026-05-11T14:30:16,495 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml' 2026-05-11T14:30:16,496 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,498 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml' 2026-05-11T14:30:16,499 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml' 2026-05-11T14:30:16,500 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,501 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,503 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml' 2026-05-11T14:30:16,504 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,505 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,506 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,507 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,508 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml' 2026-05-11T14:30:16,510 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml' 2026-05-11T14:30:16,511 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,513 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml' 2026-05-11T14:30:16,514 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,515 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,516 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,518 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml' 2026-05-11T14:30:16,519 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,520 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,521 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,522 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,524 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,525 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,526 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,527 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,529 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,530 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,531 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,533 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,534 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,535 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,536 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml' 2026-05-11T14:30:16,537 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,538 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml' 2026-05-11T14:30:16,539 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,541 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,542 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,543 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml' 2026-05-11T14:30:16,544 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml' 2026-05-11T14:30:16,546 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,547 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,548 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,549 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,550 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml' 2026-05-11T14:30:16,552 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml' 2026-05-11T14:30:16,553 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml' 2026-05-11T14:30:16,554 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,555 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml' 2026-05-11T14:30:16,556 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,557 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,559 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml' 2026-05-11T14:30:16,560 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml' 2026-05-11T14:30:16,561 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml' 2026-05-11T14:30:16,562 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,563 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml' 2026-05-11T14:30:16,564 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml' 2026-05-11T14:30:16,567 adding 'lm_eval/tasks/mmlu/generative/_default_template_yaml' 2026-05-11T14:30:16,568 adding 'lm_eval/tasks/mmlu/generative/_mmlu.yaml' 2026-05-11T14:30:16,570 adding 'lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,571 adding 'lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml' 2026-05-11T14:30:16,572 adding 'lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml' 2026-05-11T14:30:16,573 adding 'lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,574 adding 'lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,576 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml' 2026-05-11T14:30:16,577 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,578 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,579 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,580 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,581 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml' 2026-05-11T14:30:16,582 adding 'lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml' 2026-05-11T14:30:16,583 adding 'lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,584 adding 'lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml' 2026-05-11T14:30:16,585 adding 'lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,587 adding 'lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,588 adding 'lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,589 adding 'lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml' 2026-05-11T14:30:16,590 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,591 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,593 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,594 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,595 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,596 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,597 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,598 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,599 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,601 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,602 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,603 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,604 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,605 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,606 adding 'lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml' 2026-05-11T14:30:16,608 adding 'lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,609 adding 'lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml' 2026-05-11T14:30:16,610 adding 'lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,611 adding 'lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,612 adding 'lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,613 adding 'lm_eval/tasks/mmlu/generative/mmlu_management.yaml' 2026-05-11T14:30:16,615 adding 'lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml' 2026-05-11T14:30:16,616 adding 'lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,617 adding 'lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,618 adding 'lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,619 adding 'lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,621 adding 'lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml' 2026-05-11T14:30:16,622 adding 'lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml' 2026-05-11T14:30:16,623 adding 'lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml' 2026-05-11T14:30:16,624 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,625 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml' 2026-05-11T14:30:16,627 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,628 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,629 adding 'lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml' 2026-05-11T14:30:16,630 adding 'lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml' 2026-05-11T14:30:16,631 adding 'lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml' 2026-05-11T14:30:16,632 adding 'lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,633 adding 'lm_eval/tasks/mmlu/generative/mmlu_virology.yaml' 2026-05-11T14:30:16,634 adding 'lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml' 2026-05-11T14:30:16,636 adding 'lm_eval/tasks/mmlu-pro-plus/README.md' 2026-05-11T14:30:16,637 adding 'lm_eval/tasks/mmlu-pro-plus/_default_template_yaml' 2026-05-11T14:30:16,639 adding 'lm_eval/tasks/mmlu-pro-plus/_mmlu_pro_plus.yaml' 2026-05-11T14:30:16,640 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_biology.yaml' 2026-05-11T14:30:16,641 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_business.yaml' 2026-05-11T14:30:16,642 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_chemistry.yaml' 2026-05-11T14:30:16,644 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_computer_science.yaml' 2026-05-11T14:30:16,645 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_economics.yaml' 2026-05-11T14:30:16,646 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_engineering.yaml' 2026-05-11T14:30:16,647 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_health.yaml' 2026-05-11T14:30:16,648 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_history.yaml' 2026-05-11T14:30:16,649 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_law.yaml' 2026-05-11T14:30:16,651 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_math.yaml' 2026-05-11T14:30:16,652 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_other.yaml' 2026-05-11T14:30:16,653 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_philosophy.yaml' 2026-05-11T14:30:16,654 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_physics.yaml' 2026-05-11T14:30:16,655 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_psychology.yaml' 2026-05-11T14:30:16,656 adding 'lm_eval/tasks/mmlu-pro-plus/utils.py' 2026-05-11T14:30:16,660 adding 'lm_eval/tasks/mmlu-redux/generative/README.md' 2026-05-11T14:30:16,661 adding 'lm_eval/tasks/mmlu-redux/generative/_default_template_yaml' 2026-05-11T14:30:16,662 adding 'lm_eval/tasks/mmlu-redux/generative/_mmlu.yaml' 2026-05-11T14:30:16,663 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,664 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_anatomy.yaml' 2026-05-11T14:30:16,665 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_astronomy.yaml' 2026-05-11T14:30:16,667 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,668 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,669 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_biology.yaml' 2026-05-11T14:30:16,670 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,671 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,672 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,674 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,675 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_physics.yaml' 2026-05-11T14:30:16,676 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_computer_security.yaml' 2026-05-11T14:30:16,677 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,678 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_econometrics.yaml' 2026-05-11T14:30:16,679 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,681 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,682 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,683 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_global_facts.yaml' 2026-05-11T14:30:16,684 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,685 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,686 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,688 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,689 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,690 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,691 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,692 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,693 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,694 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,696 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,697 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,698 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,699 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,700 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_human_aging.yaml' 2026-05-11T14:30:16,701 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,703 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_international_law.yaml' 2026-05-11T14:30:16,704 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,705 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,706 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,707 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_management.yaml' 2026-05-11T14:30:16,708 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_marketing.yaml' 2026-05-11T14:30:16,709 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,711 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,712 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,713 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,714 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_nutrition.yaml' 2026-05-11T14:30:16,715 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_philosophy.yaml' 2026-05-11T14:30:16,717 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_prehistory.yaml' 2026-05-11T14:30:16,718 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,719 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_law.yaml' 2026-05-11T14:30:16,720 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,722 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,723 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_public_relations.yaml' 2026-05-11T14:30:16,724 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_security_studies.yaml' 2026-05-11T14:30:16,725 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_sociology.yaml' 2026-05-11T14:30:16,727 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,728 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_virology.yaml' 2026-05-11T14:30:16,729 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_world_religions.yaml' 2026-05-11T14:30:16,731 adding 'lm_eval/tasks/mmlu-redux-spanish/README.md' 2026-05-11T14:30:16,732 adding 'lm_eval/tasks/mmlu-redux-spanish/mmlu-redux-2.0-spanish.yaml' 2026-05-11T14:30:16,735 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/_default_template_spanish_yaml' 2026-05-11T14:30:16,736 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/_mmlu.yaml' 2026-05-11T14:30:16,737 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_abstract_algebra.yaml' 2026-05-11T14:30:16,738 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_anatomy.yaml' 2026-05-11T14:30:16,740 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_astronomy.yaml' 2026-05-11T14:30:16,741 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_business_ethics.yaml' 2026-05-11T14:30:16,742 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T14:30:16,743 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_biology.yaml' 2026-05-11T14:30:16,744 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_chemistry.yaml' 2026-05-11T14:30:16,746 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_computer_science.yaml' 2026-05-11T14:30:16,747 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_mathematics.yaml' 2026-05-11T14:30:16,748 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_medicine.yaml' 2026-05-11T14:30:16,749 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_physics.yaml' 2026-05-11T14:30:16,750 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_computer_security.yaml' 2026-05-11T14:30:16,751 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_conceptual_physics.yaml' 2026-05-11T14:30:16,753 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_econometrics.yaml' 2026-05-11T14:30:16,754 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_electrical_engineering.yaml' 2026-05-11T14:30:16,755 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T14:30:16,756 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_formal_logic.yaml' 2026-05-11T14:30:16,757 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_global_facts.yaml' 2026-05-11T14:30:16,758 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_biology.yaml' 2026-05-11T14:30:16,760 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T14:30:16,761 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T14:30:16,762 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_european_history.yaml' 2026-05-11T14:30:16,763 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_geography.yaml' 2026-05-11T14:30:16,764 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T14:30:16,765 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T14:30:16,767 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T14:30:16,768 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T14:30:16,769 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_physics.yaml' 2026-05-11T14:30:16,770 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_psychology.yaml' 2026-05-11T14:30:16,771 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_statistics.yaml' 2026-05-11T14:30:16,773 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_us_history.yaml' 2026-05-11T14:30:16,774 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_world_history.yaml' 2026-05-11T14:30:16,775 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_aging.yaml' 2026-05-11T14:30:16,776 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_sexuality.yaml' 2026-05-11T14:30:16,777 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_international_law.yaml' 2026-05-11T14:30:16,778 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_jurisprudence.yaml' 2026-05-11T14:30:16,779 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_logical_fallacies.yaml' 2026-05-11T14:30:16,780 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_machine_learning.yaml' 2026-05-11T14:30:16,782 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_management.yaml' 2026-05-11T14:30:16,783 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_marketing.yaml' 2026-05-11T14:30:16,784 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_medical_genetics.yaml' 2026-05-11T14:30:16,785 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_miscellaneous.yaml' 2026-05-11T14:30:16,786 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_disputes.yaml' 2026-05-11T14:30:16,787 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_scenarios.yaml' 2026-05-11T14:30:16,788 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_nutrition.yaml' 2026-05-11T14:30:16,790 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_philosophy.yaml' 2026-05-11T14:30:16,791 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_prehistory.yaml' 2026-05-11T14:30:16,792 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_accounting.yaml' 2026-05-11T14:30:16,793 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_law.yaml' 2026-05-11T14:30:16,794 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_medicine.yaml' 2026-05-11T14:30:16,795 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_psychology.yaml' 2026-05-11T14:30:16,796 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_public_relations.yaml' 2026-05-11T14:30:16,798 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_security_studies.yaml' 2026-05-11T14:30:16,799 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_sociology.yaml' 2026-05-11T14:30:16,800 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T14:30:16,801 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_virology.yaml' 2026-05-11T14:30:16,802 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_world_religions.yaml' 2026-05-11T14:30:16,805 adding 'lm_eval/tasks/mmlu_pro/README.md' 2026-05-11T14:30:16,806 adding 'lm_eval/tasks/mmlu_pro/_default_template_yaml' 2026-05-11T14:30:16,807 adding 'lm_eval/tasks/mmlu_pro/_mmlu_pro.yaml' 2026-05-11T14:30:16,808 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_biology.yaml' 2026-05-11T14:30:16,809 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_business.yaml' 2026-05-11T14:30:16,811 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_chemistry.yaml' 2026-05-11T14:30:16,812 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_computer_science.yaml' 2026-05-11T14:30:16,813 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_economics.yaml' 2026-05-11T14:30:16,814 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_engineering.yaml' 2026-05-11T14:30:16,815 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_health.yaml' 2026-05-11T14:30:16,817 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_history.yaml' 2026-05-11T14:30:16,818 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_law.yaml' 2026-05-11T14:30:16,819 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_math.yaml' 2026-05-11T14:30:16,820 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_other.yaml' 2026-05-11T14:30:16,821 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_philosophy.yaml' 2026-05-11T14:30:16,822 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_physics.yaml' 2026-05-11T14:30:16,824 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_psychology.yaml' 2026-05-11T14:30:16,825 adding 'lm_eval/tasks/mmlu_pro/utils.py' 2026-05-11T14:30:16,828 adding 'lm_eval/tasks/mmlu_prox/README.md' 2026-05-11T14:30:16,831 adding 'lm_eval/tasks/mmlu_prox/lang_libs.py' 2026-05-11T14:30:16,832 adding 'lm_eval/tasks/mmlu_prox/mmlu_prox_config_generator.py' 2026-05-11T14:30:16,834 adding 'lm_eval/tasks/mmlu_prox/mmlu_prox_lite_config_generator.py' 2026-05-11T14:30:16,836 adding 'lm_eval/tasks/mmlu_prox/af/_af_lite_template_yaml' 2026-05-11T14:30:16,837 adding 'lm_eval/tasks/mmlu_prox/af/_af_template_yaml' 2026-05-11T14:30:16,838 adding 'lm_eval/tasks/mmlu_prox/af/_mmlu_prox_af.yaml' 2026-05-11T14:30:16,839 adding 'lm_eval/tasks/mmlu_prox/af/_mmlu_prox_lite_af.yaml' 2026-05-11T14:30:16,841 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_biology.yaml' 2026-05-11T14:30:16,842 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_business.yaml' 2026-05-11T14:30:16,843 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_chemistry.yaml' 2026-05-11T14:30:16,844 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_computer_science.yaml' 2026-05-11T14:30:16,845 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_economics.yaml' 2026-05-11T14:30:16,846 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_engineering.yaml' 2026-05-11T14:30:16,847 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_health.yaml' 2026-05-11T14:30:16,848 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_history.yaml' 2026-05-11T14:30:16,849 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_law.yaml' 2026-05-11T14:30:16,851 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_math.yaml' 2026-05-11T14:30:16,852 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_other.yaml' 2026-05-11T14:30:16,853 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_philosophy.yaml' 2026-05-11T14:30:16,854 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_physics.yaml' 2026-05-11T14:30:16,855 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_psychology.yaml' 2026-05-11T14:30:16,856 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_biology.yaml' 2026-05-11T14:30:16,857 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_business.yaml' 2026-05-11T14:30:16,859 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_chemistry.yaml' 2026-05-11T14:30:16,860 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_computer_science.yaml' 2026-05-11T14:30:16,861 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_economics.yaml' 2026-05-11T14:30:16,862 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_engineering.yaml' 2026-05-11T14:30:16,863 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_health.yaml' 2026-05-11T14:30:16,865 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_history.yaml' 2026-05-11T14:30:16,866 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_law.yaml' 2026-05-11T14:30:16,867 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_math.yaml' 2026-05-11T14:30:16,868 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_other.yaml' 2026-05-11T14:30:16,869 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_philosophy.yaml' 2026-05-11T14:30:16,870 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_physics.yaml' 2026-05-11T14:30:16,872 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_psychology.yaml' 2026-05-11T14:30:16,873 adding 'lm_eval/tasks/mmlu_prox/af/utils.py' 2026-05-11T14:30:16,875 adding 'lm_eval/tasks/mmlu_prox/ar/_ar_lite_template_yaml' 2026-05-11T14:30:16,877 adding 'lm_eval/tasks/mmlu_prox/ar/_ar_template_yaml' 2026-05-11T14:30:16,878 adding 'lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_ar.yaml' 2026-05-11T14:30:16,879 adding 'lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_lite_ar.yaml' 2026-05-11T14:30:16,880 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_biology.yaml' 2026-05-11T14:30:16,881 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_business.yaml' 2026-05-11T14:30:16,883 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_chemistry.yaml' 2026-05-11T14:30:16,884 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_computer_science.yaml' 2026-05-11T14:30:16,885 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_economics.yaml' 2026-05-11T14:30:16,886 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_engineering.yaml' 2026-05-11T14:30:16,887 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_health.yaml' 2026-05-11T14:30:16,889 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_history.yaml' 2026-05-11T14:30:16,890 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_law.yaml' 2026-05-11T14:30:16,891 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_math.yaml' 2026-05-11T14:30:16,892 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_other.yaml' 2026-05-11T14:30:16,893 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_philosophy.yaml' 2026-05-11T14:30:16,894 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_physics.yaml' 2026-05-11T14:30:16,895 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_psychology.yaml' 2026-05-11T14:30:16,896 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_biology.yaml' 2026-05-11T14:30:16,897 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_business.yaml' 2026-05-11T14:30:16,899 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_chemistry.yaml' 2026-05-11T14:30:16,900 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_computer_science.yaml' 2026-05-11T14:30:16,901 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_economics.yaml' 2026-05-11T14:30:16,902 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_engineering.yaml' 2026-05-11T14:30:16,903 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_health.yaml' 2026-05-11T14:30:16,905 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_history.yaml' 2026-05-11T14:30:16,906 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_law.yaml' 2026-05-11T14:30:16,907 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_math.yaml' 2026-05-11T14:30:16,908 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_other.yaml' 2026-05-11T14:30:16,909 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_philosophy.yaml' 2026-05-11T14:30:16,910 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_physics.yaml' 2026-05-11T14:30:16,912 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_psychology.yaml' 2026-05-11T14:30:16,913 adding 'lm_eval/tasks/mmlu_prox/ar/utils.py' 2026-05-11T14:30:16,915 adding 'lm_eval/tasks/mmlu_prox/bn/_bn_lite_template_yaml' 2026-05-11T14:30:16,916 adding 'lm_eval/tasks/mmlu_prox/bn/_bn_template_yaml' 2026-05-11T14:30:16,918 adding 'lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_bn.yaml' 2026-05-11T14:30:16,919 adding 'lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_lite_bn.yaml' 2026-05-11T14:30:16,920 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_biology.yaml' 2026-05-11T14:30:16,921 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_business.yaml' 2026-05-11T14:30:16,922 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_chemistry.yaml' 2026-05-11T14:30:16,923 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_computer_science.yaml' 2026-05-11T14:30:16,925 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_economics.yaml' 2026-05-11T14:30:16,926 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_engineering.yaml' 2026-05-11T14:30:16,927 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_health.yaml' 2026-05-11T14:30:16,928 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_history.yaml' 2026-05-11T14:30:16,929 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_law.yaml' 2026-05-11T14:30:16,931 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_math.yaml' 2026-05-11T14:30:16,932 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_other.yaml' 2026-05-11T14:30:16,933 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_philosophy.yaml' 2026-05-11T14:30:16,934 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_physics.yaml' 2026-05-11T14:30:16,935 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_psychology.yaml' 2026-05-11T14:30:16,937 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_biology.yaml' 2026-05-11T14:30:16,938 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_business.yaml' 2026-05-11T14:30:16,939 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_chemistry.yaml' 2026-05-11T14:30:16,940 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_computer_science.yaml' 2026-05-11T14:30:16,942 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_economics.yaml' 2026-05-11T14:30:16,943 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_engineering.yaml' 2026-05-11T14:30:16,944 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_health.yaml' 2026-05-11T14:30:16,945 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_history.yaml' 2026-05-11T14:30:16,947 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_law.yaml' 2026-05-11T14:30:16,948 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_math.yaml' 2026-05-11T14:30:16,949 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_other.yaml' 2026-05-11T14:30:16,950 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_philosophy.yaml' 2026-05-11T14:30:16,951 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_physics.yaml' 2026-05-11T14:30:16,952 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_psychology.yaml' 2026-05-11T14:30:16,953 adding 'lm_eval/tasks/mmlu_prox/bn/utils.py' 2026-05-11T14:30:16,956 adding 'lm_eval/tasks/mmlu_prox/cs/_cs_lite_template_yaml' 2026-05-11T14:30:16,957 adding 'lm_eval/tasks/mmlu_prox/cs/_cs_template_yaml' 2026-05-11T14:30:16,958 adding 'lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_cs.yaml' 2026-05-11T14:30:16,959 adding 'lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_lite_cs.yaml' 2026-05-11T14:30:16,960 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_biology.yaml' 2026-05-11T14:30:16,961 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_business.yaml' 2026-05-11T14:30:16,963 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_chemistry.yaml' 2026-05-11T14:30:16,964 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_computer_science.yaml' 2026-05-11T14:30:16,965 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_economics.yaml' 2026-05-11T14:30:16,966 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_engineering.yaml' 2026-05-11T14:30:16,967 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_health.yaml' 2026-05-11T14:30:16,968 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_history.yaml' 2026-05-11T14:30:16,969 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_law.yaml' 2026-05-11T14:30:16,971 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_math.yaml' 2026-05-11T14:30:16,972 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_other.yaml' 2026-05-11T14:30:16,973 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_philosophy.yaml' 2026-05-11T14:30:16,974 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_physics.yaml' 2026-05-11T14:30:16,975 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_psychology.yaml' 2026-05-11T14:30:16,976 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_biology.yaml' 2026-05-11T14:30:16,978 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_business.yaml' 2026-05-11T14:30:16,979 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_chemistry.yaml' 2026-05-11T14:30:16,980 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_computer_science.yaml' 2026-05-11T14:30:16,981 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_economics.yaml' 2026-05-11T14:30:16,982 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_engineering.yaml' 2026-05-11T14:30:16,983 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_health.yaml' 2026-05-11T14:30:16,985 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_history.yaml' 2026-05-11T14:30:16,986 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_law.yaml' 2026-05-11T14:30:16,987 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_math.yaml' 2026-05-11T14:30:16,988 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_other.yaml' 2026-05-11T14:30:16,989 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_philosophy.yaml' 2026-05-11T14:30:16,991 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_physics.yaml' 2026-05-11T14:30:16,992 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_psychology.yaml' 2026-05-11T14:30:16,993 adding 'lm_eval/tasks/mmlu_prox/cs/utils.py' 2026-05-11T14:30:16,995 adding 'lm_eval/tasks/mmlu_prox/de/_de_lite_template_yaml' 2026-05-11T14:30:16,997 adding 'lm_eval/tasks/mmlu_prox/de/_de_template_yaml' 2026-05-11T14:30:16,998 adding 'lm_eval/tasks/mmlu_prox/de/_mmlu_prox_de.yaml' 2026-05-11T14:30:16,999 adding 'lm_eval/tasks/mmlu_prox/de/_mmlu_prox_lite_de.yaml' 2026-05-11T14:30:17,000 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_biology.yaml' 2026-05-11T14:30:17,002 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_business.yaml' 2026-05-11T14:30:17,003 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_chemistry.yaml' 2026-05-11T14:30:17,004 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_computer_science.yaml' 2026-05-11T14:30:17,005 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_economics.yaml' 2026-05-11T14:30:17,006 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_engineering.yaml' 2026-05-11T14:30:17,007 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_health.yaml' 2026-05-11T14:30:17,008 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_history.yaml' 2026-05-11T14:30:17,009 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_law.yaml' 2026-05-11T14:30:17,010 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_math.yaml' 2026-05-11T14:30:17,011 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_other.yaml' 2026-05-11T14:30:17,012 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_philosophy.yaml' 2026-05-11T14:30:17,013 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_physics.yaml' 2026-05-11T14:30:17,015 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_psychology.yaml' 2026-05-11T14:30:17,016 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_biology.yaml' 2026-05-11T14:30:17,017 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_business.yaml' 2026-05-11T14:30:17,018 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_chemistry.yaml' 2026-05-11T14:30:17,020 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_computer_science.yaml' 2026-05-11T14:30:17,021 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_economics.yaml' 2026-05-11T14:30:17,022 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_engineering.yaml' 2026-05-11T14:30:17,023 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_health.yaml' 2026-05-11T14:30:17,024 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_history.yaml' 2026-05-11T14:30:17,025 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_law.yaml' 2026-05-11T14:30:17,027 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_math.yaml' 2026-05-11T14:30:17,028 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_other.yaml' 2026-05-11T14:30:17,029 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_philosophy.yaml' 2026-05-11T14:30:17,030 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_physics.yaml' 2026-05-11T14:30:17,031 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_psychology.yaml' 2026-05-11T14:30:17,032 adding 'lm_eval/tasks/mmlu_prox/de/utils.py' 2026-05-11T14:30:17,035 adding 'lm_eval/tasks/mmlu_prox/en/_en_lite_template_yaml' 2026-05-11T14:30:17,036 adding 'lm_eval/tasks/mmlu_prox/en/_en_template_yaml' 2026-05-11T14:30:17,037 adding 'lm_eval/tasks/mmlu_prox/en/_mmlu_prox_en.yaml' 2026-05-11T14:30:17,038 adding 'lm_eval/tasks/mmlu_prox/en/_mmlu_prox_lite_en.yaml' 2026-05-11T14:30:17,039 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_biology.yaml' 2026-05-11T14:30:17,041 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_business.yaml' 2026-05-11T14:30:17,042 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_chemistry.yaml' 2026-05-11T14:30:17,043 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_computer_science.yaml' 2026-05-11T14:30:17,044 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_economics.yaml' 2026-05-11T14:30:17,045 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_engineering.yaml' 2026-05-11T14:30:17,047 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_health.yaml' 2026-05-11T14:30:17,048 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_history.yaml' 2026-05-11T14:30:17,049 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_law.yaml' 2026-05-11T14:30:17,050 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_math.yaml' 2026-05-11T14:30:17,051 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_other.yaml' 2026-05-11T14:30:17,052 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_philosophy.yaml' 2026-05-11T14:30:17,054 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_physics.yaml' 2026-05-11T14:30:17,055 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_psychology.yaml' 2026-05-11T14:30:17,056 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_biology.yaml' 2026-05-11T14:30:17,057 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_business.yaml' 2026-05-11T14:30:17,058 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_chemistry.yaml' 2026-05-11T14:30:17,059 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_computer_science.yaml' 2026-05-11T14:30:17,060 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_economics.yaml' 2026-05-11T14:30:17,061 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_engineering.yaml' 2026-05-11T14:30:17,062 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_health.yaml' 2026-05-11T14:30:17,063 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_history.yaml' 2026-05-11T14:30:17,064 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_law.yaml' 2026-05-11T14:30:17,065 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_math.yaml' 2026-05-11T14:30:17,067 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_other.yaml' 2026-05-11T14:30:17,068 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_philosophy.yaml' 2026-05-11T14:30:17,069 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_physics.yaml' 2026-05-11T14:30:17,070 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_psychology.yaml' 2026-05-11T14:30:17,071 adding 'lm_eval/tasks/mmlu_prox/en/utils.py' 2026-05-11T14:30:17,074 adding 'lm_eval/tasks/mmlu_prox/es/_es_lite_template_yaml' 2026-05-11T14:30:17,075 adding 'lm_eval/tasks/mmlu_prox/es/_es_template_yaml' 2026-05-11T14:30:17,076 adding 'lm_eval/tasks/mmlu_prox/es/_mmlu_prox_es.yaml' 2026-05-11T14:30:17,077 adding 'lm_eval/tasks/mmlu_prox/es/_mmlu_prox_lite_es.yaml' 2026-05-11T14:30:17,078 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_biology.yaml' 2026-05-11T14:30:17,080 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_business.yaml' 2026-05-11T14:30:17,081 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_chemistry.yaml' 2026-05-11T14:30:17,082 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_computer_science.yaml' 2026-05-11T14:30:17,083 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_economics.yaml' 2026-05-11T14:30:17,084 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_engineering.yaml' 2026-05-11T14:30:17,085 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_health.yaml' 2026-05-11T14:30:17,087 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_history.yaml' 2026-05-11T14:30:17,088 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_law.yaml' 2026-05-11T14:30:17,089 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_math.yaml' 2026-05-11T14:30:17,090 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_other.yaml' 2026-05-11T14:30:17,091 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_philosophy.yaml' 2026-05-11T14:30:17,092 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_physics.yaml' 2026-05-11T14:30:17,094 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_psychology.yaml' 2026-05-11T14:30:17,095 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_biology.yaml' 2026-05-11T14:30:17,096 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_business.yaml' 2026-05-11T14:30:17,097 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_chemistry.yaml' 2026-05-11T14:30:17,098 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_computer_science.yaml' 2026-05-11T14:30:17,099 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_economics.yaml' 2026-05-11T14:30:17,101 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_engineering.yaml' 2026-05-11T14:30:17,102 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_health.yaml' 2026-05-11T14:30:17,103 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_history.yaml' 2026-05-11T14:30:17,104 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_law.yaml' 2026-05-11T14:30:17,105 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_math.yaml' 2026-05-11T14:30:17,107 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_other.yaml' 2026-05-11T14:30:17,108 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_philosophy.yaml' 2026-05-11T14:30:17,109 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_physics.yaml' 2026-05-11T14:30:17,110 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_psychology.yaml' 2026-05-11T14:30:17,111 adding 'lm_eval/tasks/mmlu_prox/es/utils.py' 2026-05-11T14:30:17,113 adding 'lm_eval/tasks/mmlu_prox/fr/_fr_lite_template_yaml' 2026-05-11T14:30:17,114 adding 'lm_eval/tasks/mmlu_prox/fr/_fr_template_yaml' 2026-05-11T14:30:17,115 adding 'lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_fr.yaml' 2026-05-11T14:30:17,117 adding 'lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_lite_fr.yaml' 2026-05-11T14:30:17,118 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_biology.yaml' 2026-05-11T14:30:17,119 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_business.yaml' 2026-05-11T14:30:17,120 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_chemistry.yaml' 2026-05-11T14:30:17,121 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_computer_science.yaml' 2026-05-11T14:30:17,122 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_economics.yaml' 2026-05-11T14:30:17,123 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_engineering.yaml' 2026-05-11T14:30:17,124 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_health.yaml' 2026-05-11T14:30:17,126 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_history.yaml' 2026-05-11T14:30:17,127 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_law.yaml' 2026-05-11T14:30:17,128 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_math.yaml' 2026-05-11T14:30:17,129 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_other.yaml' 2026-05-11T14:30:17,130 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_philosophy.yaml' 2026-05-11T14:30:17,131 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_physics.yaml' 2026-05-11T14:30:17,132 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_psychology.yaml' 2026-05-11T14:30:17,134 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_biology.yaml' 2026-05-11T14:30:17,135 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_business.yaml' 2026-05-11T14:30:17,136 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_chemistry.yaml' 2026-05-11T14:30:17,137 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_computer_science.yaml' 2026-05-11T14:30:17,139 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_economics.yaml' 2026-05-11T14:30:17,140 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_engineering.yaml' 2026-05-11T14:30:17,141 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_health.yaml' 2026-05-11T14:30:17,142 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_history.yaml' 2026-05-11T14:30:17,143 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_law.yaml' 2026-05-11T14:30:17,145 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_math.yaml' 2026-05-11T14:30:17,146 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_other.yaml' 2026-05-11T14:30:17,147 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_philosophy.yaml' 2026-05-11T14:30:17,148 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_physics.yaml' 2026-05-11T14:30:17,149 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_psychology.yaml' 2026-05-11T14:30:17,151 adding 'lm_eval/tasks/mmlu_prox/fr/utils.py' 2026-05-11T14:30:17,153 adding 'lm_eval/tasks/mmlu_prox/hi/_hi_lite_template_yaml' 2026-05-11T14:30:17,154 adding 'lm_eval/tasks/mmlu_prox/hi/_hi_template_yaml' 2026-05-11T14:30:17,155 adding 'lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_hi.yaml' 2026-05-11T14:30:17,157 adding 'lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_lite_hi.yaml' 2026-05-11T14:30:17,158 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_biology.yaml' 2026-05-11T14:30:17,159 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_business.yaml' 2026-05-11T14:30:17,160 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_chemistry.yaml' 2026-05-11T14:30:17,162 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_computer_science.yaml' 2026-05-11T14:30:17,163 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_economics.yaml' 2026-05-11T14:30:17,164 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_engineering.yaml' 2026-05-11T14:30:17,165 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_health.yaml' 2026-05-11T14:30:17,166 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_history.yaml' 2026-05-11T14:30:17,167 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_law.yaml' 2026-05-11T14:30:17,169 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_math.yaml' 2026-05-11T14:30:17,170 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_other.yaml' 2026-05-11T14:30:17,171 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_philosophy.yaml' 2026-05-11T14:30:17,172 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_physics.yaml' 2026-05-11T14:30:17,173 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_psychology.yaml' 2026-05-11T14:30:17,174 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_biology.yaml' 2026-05-11T14:30:17,175 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_business.yaml' 2026-05-11T14:30:17,176 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_chemistry.yaml' 2026-05-11T14:30:17,178 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_computer_science.yaml' 2026-05-11T14:30:17,179 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_economics.yaml' 2026-05-11T14:30:17,180 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_engineering.yaml' 2026-05-11T14:30:17,181 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_health.yaml' 2026-05-11T14:30:17,182 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_history.yaml' 2026-05-11T14:30:17,183 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_law.yaml' 2026-05-11T14:30:17,185 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_math.yaml' 2026-05-11T14:30:17,186 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_other.yaml' 2026-05-11T14:30:17,187 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_philosophy.yaml' 2026-05-11T14:30:17,188 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_physics.yaml' 2026-05-11T14:30:17,189 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_psychology.yaml' 2026-05-11T14:30:17,191 adding 'lm_eval/tasks/mmlu_prox/hi/utils.py' 2026-05-11T14:30:17,193 adding 'lm_eval/tasks/mmlu_prox/hu/_hu_lite_template_yaml' 2026-05-11T14:30:17,194 adding 'lm_eval/tasks/mmlu_prox/hu/_hu_template_yaml' 2026-05-11T14:30:17,195 adding 'lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_hu.yaml' 2026-05-11T14:30:17,197 adding 'lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_lite_hu.yaml' 2026-05-11T14:30:17,198 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_biology.yaml' 2026-05-11T14:30:17,199 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_business.yaml' 2026-05-11T14:30:17,200 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_chemistry.yaml' 2026-05-11T14:30:17,202 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_computer_science.yaml' 2026-05-11T14:30:17,203 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_economics.yaml' 2026-05-11T14:30:17,204 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_engineering.yaml' 2026-05-11T14:30:17,205 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_health.yaml' 2026-05-11T14:30:17,206 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_history.yaml' 2026-05-11T14:30:17,208 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_law.yaml' 2026-05-11T14:30:17,209 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_math.yaml' 2026-05-11T14:30:17,210 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_other.yaml' 2026-05-11T14:30:17,211 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_philosophy.yaml' 2026-05-11T14:30:17,212 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_physics.yaml' 2026-05-11T14:30:17,214 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_psychology.yaml' 2026-05-11T14:30:17,215 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_biology.yaml' 2026-05-11T14:30:17,216 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_business.yaml' 2026-05-11T14:30:17,217 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_chemistry.yaml' 2026-05-11T14:30:17,219 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_computer_science.yaml' 2026-05-11T14:30:17,220 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_economics.yaml' 2026-05-11T14:30:17,221 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_engineering.yaml' 2026-05-11T14:30:17,222 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_health.yaml' 2026-05-11T14:30:17,224 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_history.yaml' 2026-05-11T14:30:17,225 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_law.yaml' 2026-05-11T14:30:17,226 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_math.yaml' 2026-05-11T14:30:17,227 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_other.yaml' 2026-05-11T14:30:17,228 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_philosophy.yaml' 2026-05-11T14:30:17,230 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_physics.yaml' 2026-05-11T14:30:17,231 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_psychology.yaml' 2026-05-11T14:30:17,232 adding 'lm_eval/tasks/mmlu_prox/hu/utils.py' 2026-05-11T14:30:17,234 adding 'lm_eval/tasks/mmlu_prox/id/_id_lite_template_yaml' 2026-05-11T14:30:17,235 adding 'lm_eval/tasks/mmlu_prox/id/_id_template_yaml' 2026-05-11T14:30:17,237 adding 'lm_eval/tasks/mmlu_prox/id/_mmlu_prox_id.yaml' 2026-05-11T14:30:17,238 adding 'lm_eval/tasks/mmlu_prox/id/_mmlu_prox_lite_id.yaml' 2026-05-11T14:30:17,239 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_biology.yaml' 2026-05-11T14:30:17,240 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_business.yaml' 2026-05-11T14:30:17,241 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_chemistry.yaml' 2026-05-11T14:30:17,243 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_computer_science.yaml' 2026-05-11T14:30:17,244 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_economics.yaml' 2026-05-11T14:30:17,245 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_engineering.yaml' 2026-05-11T14:30:17,246 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_health.yaml' 2026-05-11T14:30:17,247 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_history.yaml' 2026-05-11T14:30:17,248 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_law.yaml' 2026-05-11T14:30:17,250 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_math.yaml' 2026-05-11T14:30:17,251 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_other.yaml' 2026-05-11T14:30:17,252 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_philosophy.yaml' 2026-05-11T14:30:17,253 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_physics.yaml' 2026-05-11T14:30:17,254 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_psychology.yaml' 2026-05-11T14:30:17,255 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_biology.yaml' 2026-05-11T14:30:17,257 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_business.yaml' 2026-05-11T14:30:17,258 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_chemistry.yaml' 2026-05-11T14:30:17,259 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_computer_science.yaml' 2026-05-11T14:30:17,260 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_economics.yaml' 2026-05-11T14:30:17,261 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_engineering.yaml' 2026-05-11T14:30:17,263 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_health.yaml' 2026-05-11T14:30:17,264 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_history.yaml' 2026-05-11T14:30:17,265 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_law.yaml' 2026-05-11T14:30:17,266 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_math.yaml' 2026-05-11T14:30:17,267 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_other.yaml' 2026-05-11T14:30:17,268 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_philosophy.yaml' 2026-05-11T14:30:17,269 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_physics.yaml' 2026-05-11T14:30:17,271 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_psychology.yaml' 2026-05-11T14:30:17,272 adding 'lm_eval/tasks/mmlu_prox/id/utils.py' 2026-05-11T14:30:17,274 adding 'lm_eval/tasks/mmlu_prox/it/_it_lite_template_yaml' 2026-05-11T14:30:17,276 adding 'lm_eval/tasks/mmlu_prox/it/_it_template_yaml' 2026-05-11T14:30:17,277 adding 'lm_eval/tasks/mmlu_prox/it/_mmlu_prox_it.yaml' 2026-05-11T14:30:17,278 adding 'lm_eval/tasks/mmlu_prox/it/_mmlu_prox_lite_it.yaml' 2026-05-11T14:30:17,279 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_biology.yaml' 2026-05-11T14:30:17,280 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_business.yaml' 2026-05-11T14:30:17,281 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_chemistry.yaml' 2026-05-11T14:30:17,282 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_computer_science.yaml' 2026-05-11T14:30:17,283 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_economics.yaml' 2026-05-11T14:30:17,284 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_engineering.yaml' 2026-05-11T14:30:17,285 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_health.yaml' 2026-05-11T14:30:17,286 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_history.yaml' 2026-05-11T14:30:17,287 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_law.yaml' 2026-05-11T14:30:17,289 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_math.yaml' 2026-05-11T14:30:17,290 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_other.yaml' 2026-05-11T14:30:17,291 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_philosophy.yaml' 2026-05-11T14:30:17,292 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_physics.yaml' 2026-05-11T14:30:17,293 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_psychology.yaml' 2026-05-11T14:30:17,294 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_biology.yaml' 2026-05-11T14:30:17,296 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_business.yaml' 2026-05-11T14:30:17,298 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_chemistry.yaml' 2026-05-11T14:30:17,300 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_computer_science.yaml' 2026-05-11T14:30:17,301 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_economics.yaml' 2026-05-11T14:30:17,302 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_engineering.yaml' 2026-05-11T14:30:17,303 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_health.yaml' 2026-05-11T14:30:17,304 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_history.yaml' 2026-05-11T14:30:17,306 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_law.yaml' 2026-05-11T14:30:17,307 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_math.yaml' 2026-05-11T14:30:17,308 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_other.yaml' 2026-05-11T14:30:17,309 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_philosophy.yaml' 2026-05-11T14:30:17,310 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_physics.yaml' 2026-05-11T14:30:17,312 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_psychology.yaml' 2026-05-11T14:30:17,313 adding 'lm_eval/tasks/mmlu_prox/it/utils.py' 2026-05-11T14:30:17,315 adding 'lm_eval/tasks/mmlu_prox/ja/_ja_lite_template_yaml' 2026-05-11T14:30:17,316 adding 'lm_eval/tasks/mmlu_prox/ja/_ja_template_yaml' 2026-05-11T14:30:17,318 adding 'lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_ja.yaml' 2026-05-11T14:30:17,319 adding 'lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_lite_ja.yaml' 2026-05-11T14:30:17,320 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_biology.yaml' 2026-05-11T14:30:17,321 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_business.yaml' 2026-05-11T14:30:17,322 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_chemistry.yaml' 2026-05-11T14:30:17,324 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_computer_science.yaml' 2026-05-11T14:30:17,325 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_economics.yaml' 2026-05-11T14:30:17,326 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_engineering.yaml' 2026-05-11T14:30:17,327 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_health.yaml' 2026-05-11T14:30:17,328 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_history.yaml' 2026-05-11T14:30:17,329 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_law.yaml' 2026-05-11T14:30:17,331 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_math.yaml' 2026-05-11T14:30:17,332 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_other.yaml' 2026-05-11T14:30:17,333 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_philosophy.yaml' 2026-05-11T14:30:17,334 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_physics.yaml' 2026-05-11T14:30:17,335 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_psychology.yaml' 2026-05-11T14:30:17,337 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_biology.yaml' 2026-05-11T14:30:17,338 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_business.yaml' 2026-05-11T14:30:17,339 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_chemistry.yaml' 2026-05-11T14:30:17,340 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_computer_science.yaml' 2026-05-11T14:30:17,341 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_economics.yaml' 2026-05-11T14:30:17,342 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_engineering.yaml' 2026-05-11T14:30:17,344 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_health.yaml' 2026-05-11T14:30:17,345 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_history.yaml' 2026-05-11T14:30:17,346 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_law.yaml' 2026-05-11T14:30:17,347 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_math.yaml' 2026-05-11T14:30:17,349 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_other.yaml' 2026-05-11T14:30:17,350 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_philosophy.yaml' 2026-05-11T14:30:17,351 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_physics.yaml' 2026-05-11T14:30:17,352 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_psychology.yaml' 2026-05-11T14:30:17,353 adding 'lm_eval/tasks/mmlu_prox/ja/utils.py' 2026-05-11T14:30:17,356 adding 'lm_eval/tasks/mmlu_prox/ko/_ko_lite_template_yaml' 2026-05-11T14:30:17,357 adding 'lm_eval/tasks/mmlu_prox/ko/_ko_template_yaml' 2026-05-11T14:30:17,358 adding 'lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_ko.yaml' 2026-05-11T14:30:17,359 adding 'lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_lite_ko.yaml' 2026-05-11T14:30:17,360 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_biology.yaml' 2026-05-11T14:30:17,362 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_business.yaml' 2026-05-11T14:30:17,363 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_chemistry.yaml' 2026-05-11T14:30:17,365 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_computer_science.yaml' 2026-05-11T14:30:17,366 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_economics.yaml' 2026-05-11T14:30:17,368 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_engineering.yaml' 2026-05-11T14:30:17,369 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_health.yaml' 2026-05-11T14:30:17,370 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_history.yaml' 2026-05-11T14:30:17,371 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_law.yaml' 2026-05-11T14:30:17,373 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_math.yaml' 2026-05-11T14:30:17,374 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_other.yaml' 2026-05-11T14:30:17,375 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_philosophy.yaml' 2026-05-11T14:30:17,376 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_physics.yaml' 2026-05-11T14:30:17,378 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_psychology.yaml' 2026-05-11T14:30:17,379 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_biology.yaml' 2026-05-11T14:30:17,380 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_business.yaml' 2026-05-11T14:30:17,381 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_chemistry.yaml' 2026-05-11T14:30:17,383 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_computer_science.yaml' 2026-05-11T14:30:17,384 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_economics.yaml' 2026-05-11T14:30:17,385 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_engineering.yaml' 2026-05-11T14:30:17,386 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_health.yaml' 2026-05-11T14:30:17,388 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_history.yaml' 2026-05-11T14:30:17,389 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_law.yaml' 2026-05-11T14:30:17,390 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_math.yaml' 2026-05-11T14:30:17,391 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_other.yaml' 2026-05-11T14:30:17,392 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_philosophy.yaml' 2026-05-11T14:30:17,393 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_physics.yaml' 2026-05-11T14:30:17,395 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_psychology.yaml' 2026-05-11T14:30:17,396 adding 'lm_eval/tasks/mmlu_prox/ko/utils.py' 2026-05-11T14:30:17,398 adding 'lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_lite_mr.yaml' 2026-05-11T14:30:17,400 adding 'lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_mr.yaml' 2026-05-11T14:30:17,401 adding 'lm_eval/tasks/mmlu_prox/mr/_mr_lite_template_yaml' 2026-05-11T14:30:17,402 adding 'lm_eval/tasks/mmlu_prox/mr/_mr_template_yaml' 2026-05-11T14:30:17,404 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_biology.yaml' 2026-05-11T14:30:17,405 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_business.yaml' 2026-05-11T14:30:17,407 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_chemistry.yaml' 2026-05-11T14:30:17,408 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_computer_science.yaml' 2026-05-11T14:30:17,410 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_economics.yaml' 2026-05-11T14:30:17,411 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_engineering.yaml' 2026-05-11T14:30:17,412 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_health.yaml' 2026-05-11T14:30:17,414 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_history.yaml' 2026-05-11T14:30:17,415 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_law.yaml' 2026-05-11T14:30:17,416 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_math.yaml' 2026-05-11T14:30:17,417 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_other.yaml' 2026-05-11T14:30:17,418 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_philosophy.yaml' 2026-05-11T14:30:17,420 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_physics.yaml' 2026-05-11T14:30:17,421 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_psychology.yaml' 2026-05-11T14:30:17,422 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_biology.yaml' 2026-05-11T14:30:17,423 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_business.yaml' 2026-05-11T14:30:17,424 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_chemistry.yaml' 2026-05-11T14:30:17,426 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_computer_science.yaml' 2026-05-11T14:30:17,427 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_economics.yaml' 2026-05-11T14:30:17,428 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_engineering.yaml' 2026-05-11T14:30:17,429 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_health.yaml' 2026-05-11T14:30:17,430 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_history.yaml' 2026-05-11T14:30:17,435 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_law.yaml' 2026-05-11T14:30:17,436 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_math.yaml' 2026-05-11T14:30:17,437 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_other.yaml' 2026-05-11T14:30:17,439 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_philosophy.yaml' 2026-05-11T14:30:17,440 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_physics.yaml' 2026-05-11T14:30:17,441 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_psychology.yaml' 2026-05-11T14:30:17,442 adding 'lm_eval/tasks/mmlu_prox/mr/utils.py' 2026-05-11T14:30:17,445 adding 'lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_lite_ne.yaml' 2026-05-11T14:30:17,446 adding 'lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_ne.yaml' 2026-05-11T14:30:17,447 adding 'lm_eval/tasks/mmlu_prox/ne/_ne_lite_template_yaml' 2026-05-11T14:30:17,448 adding 'lm_eval/tasks/mmlu_prox/ne/_ne_template_yaml' 2026-05-11T14:30:17,450 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_biology.yaml' 2026-05-11T14:30:17,451 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_business.yaml' 2026-05-11T14:30:17,452 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_chemistry.yaml' 2026-05-11T14:30:17,453 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_computer_science.yaml' 2026-05-11T14:30:17,455 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_economics.yaml' 2026-05-11T14:30:17,456 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_engineering.yaml' 2026-05-11T14:30:17,457 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_health.yaml' 2026-05-11T14:30:17,458 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_history.yaml' 2026-05-11T14:30:17,459 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_law.yaml' 2026-05-11T14:30:17,460 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_math.yaml' 2026-05-11T14:30:17,462 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_other.yaml' 2026-05-11T14:30:17,463 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_philosophy.yaml' 2026-05-11T14:30:17,464 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_physics.yaml' 2026-05-11T14:30:17,465 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_psychology.yaml' 2026-05-11T14:30:17,467 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_biology.yaml' 2026-05-11T14:30:17,469 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_business.yaml' 2026-05-11T14:30:17,470 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_chemistry.yaml' 2026-05-11T14:30:17,471 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_computer_science.yaml' 2026-05-11T14:30:17,472 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_economics.yaml' 2026-05-11T14:30:17,474 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_engineering.yaml' 2026-05-11T14:30:17,475 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_health.yaml' 2026-05-11T14:30:17,476 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_history.yaml' 2026-05-11T14:30:17,477 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_law.yaml' 2026-05-11T14:30:17,479 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_math.yaml' 2026-05-11T14:30:17,480 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_other.yaml' 2026-05-11T14:30:17,481 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_philosophy.yaml' 2026-05-11T14:30:17,482 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_physics.yaml' 2026-05-11T14:30:17,484 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_psychology.yaml' 2026-05-11T14:30:17,485 adding 'lm_eval/tasks/mmlu_prox/ne/utils.py' 2026-05-11T14:30:17,487 adding 'lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_lite_pt.yaml' 2026-05-11T14:30:17,488 adding 'lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_pt.yaml' 2026-05-11T14:30:17,490 adding 'lm_eval/tasks/mmlu_prox/pt/_pt_lite_template_yaml' 2026-05-11T14:30:17,491 adding 'lm_eval/tasks/mmlu_prox/pt/_pt_template_yaml' 2026-05-11T14:30:17,492 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_biology.yaml' 2026-05-11T14:30:17,493 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_business.yaml' 2026-05-11T14:30:17,495 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_chemistry.yaml' 2026-05-11T14:30:17,496 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_computer_science.yaml' 2026-05-11T14:30:17,497 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_economics.yaml' 2026-05-11T14:30:17,498 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_engineering.yaml' 2026-05-11T14:30:17,500 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_health.yaml' 2026-05-11T14:30:17,501 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_history.yaml' 2026-05-11T14:30:17,502 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_law.yaml' 2026-05-11T14:30:17,503 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_math.yaml' 2026-05-11T14:30:17,505 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_other.yaml' 2026-05-11T14:30:17,506 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_philosophy.yaml' 2026-05-11T14:30:17,507 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_physics.yaml' 2026-05-11T14:30:17,508 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_psychology.yaml' 2026-05-11T14:30:17,509 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_biology.yaml' 2026-05-11T14:30:17,510 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_business.yaml' 2026-05-11T14:30:17,512 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_chemistry.yaml' 2026-05-11T14:30:17,513 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_computer_science.yaml' 2026-05-11T14:30:17,514 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_economics.yaml' 2026-05-11T14:30:17,515 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_engineering.yaml' 2026-05-11T14:30:17,516 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_health.yaml' 2026-05-11T14:30:17,518 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_history.yaml' 2026-05-11T14:30:17,519 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_law.yaml' 2026-05-11T14:30:17,520 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_math.yaml' 2026-05-11T14:30:17,521 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_other.yaml' 2026-05-11T14:30:17,523 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_philosophy.yaml' 2026-05-11T14:30:17,524 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_physics.yaml' 2026-05-11T14:30:17,525 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_psychology.yaml' 2026-05-11T14:30:17,527 adding 'lm_eval/tasks/mmlu_prox/pt/utils.py' 2026-05-11T14:30:17,530 adding 'lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_lite_ru.yaml' 2026-05-11T14:30:17,531 adding 'lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_ru.yaml' 2026-05-11T14:30:17,532 adding 'lm_eval/tasks/mmlu_prox/ru/_ru_lite_template_yaml' 2026-05-11T14:30:17,533 adding 'lm_eval/tasks/mmlu_prox/ru/_ru_template_yaml' 2026-05-11T14:30:17,534 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_biology.yaml' 2026-05-11T14:30:17,536 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_business.yaml' 2026-05-11T14:30:17,537 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_chemistry.yaml' 2026-05-11T14:30:17,538 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_computer_science.yaml' 2026-05-11T14:30:17,539 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_economics.yaml' 2026-05-11T14:30:17,541 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_engineering.yaml' 2026-05-11T14:30:17,542 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_health.yaml' 2026-05-11T14:30:17,543 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_history.yaml' 2026-05-11T14:30:17,544 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_law.yaml' 2026-05-11T14:30:17,545 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_math.yaml' 2026-05-11T14:30:17,546 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_other.yaml' 2026-05-11T14:30:17,548 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_philosophy.yaml' 2026-05-11T14:30:17,549 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_physics.yaml' 2026-05-11T14:30:17,550 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_psychology.yaml' 2026-05-11T14:30:17,551 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_biology.yaml' 2026-05-11T14:30:17,552 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_business.yaml' 2026-05-11T14:30:17,554 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_chemistry.yaml' 2026-05-11T14:30:17,555 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_computer_science.yaml' 2026-05-11T14:30:17,556 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_economics.yaml' 2026-05-11T14:30:17,557 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_engineering.yaml' 2026-05-11T14:30:17,559 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_health.yaml' 2026-05-11T14:30:17,560 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_history.yaml' 2026-05-11T14:30:17,561 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_law.yaml' 2026-05-11T14:30:17,562 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_math.yaml' 2026-05-11T14:30:17,563 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_other.yaml' 2026-05-11T14:30:17,565 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_philosophy.yaml' 2026-05-11T14:30:17,566 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_physics.yaml' 2026-05-11T14:30:17,567 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_psychology.yaml' 2026-05-11T14:30:17,569 adding 'lm_eval/tasks/mmlu_prox/ru/utils.py' 2026-05-11T14:30:17,571 adding 'lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_lite_sr.yaml' 2026-05-11T14:30:17,572 adding 'lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_sr.yaml' 2026-05-11T14:30:17,573 adding 'lm_eval/tasks/mmlu_prox/sr/_sr_lite_template_yaml' 2026-05-11T14:30:17,574 adding 'lm_eval/tasks/mmlu_prox/sr/_sr_template_yaml' 2026-05-11T14:30:17,576 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_biology.yaml' 2026-05-11T14:30:17,577 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_business.yaml' 2026-05-11T14:30:17,578 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_chemistry.yaml' 2026-05-11T14:30:17,579 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_computer_science.yaml' 2026-05-11T14:30:17,580 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_economics.yaml' 2026-05-11T14:30:17,581 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_engineering.yaml' 2026-05-11T14:30:17,582 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_health.yaml' 2026-05-11T14:30:17,584 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_history.yaml' 2026-05-11T14:30:17,585 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_law.yaml' 2026-05-11T14:30:17,586 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_math.yaml' 2026-05-11T14:30:17,587 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_other.yaml' 2026-05-11T14:30:17,588 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_philosophy.yaml' 2026-05-11T14:30:17,590 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_physics.yaml' 2026-05-11T14:30:17,591 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_psychology.yaml' 2026-05-11T14:30:17,592 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_biology.yaml' 2026-05-11T14:30:17,593 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_business.yaml' 2026-05-11T14:30:17,594 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_chemistry.yaml' 2026-05-11T14:30:17,596 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_computer_science.yaml' 2026-05-11T14:30:17,597 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_economics.yaml' 2026-05-11T14:30:17,598 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_engineering.yaml' 2026-05-11T14:30:17,599 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_health.yaml' 2026-05-11T14:30:17,601 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_history.yaml' 2026-05-11T14:30:17,602 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_law.yaml' 2026-05-11T14:30:17,603 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_math.yaml' 2026-05-11T14:30:17,604 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_other.yaml' 2026-05-11T14:30:17,606 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_philosophy.yaml' 2026-05-11T14:30:17,607 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_physics.yaml' 2026-05-11T14:30:17,608 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_psychology.yaml' 2026-05-11T14:30:17,609 adding 'lm_eval/tasks/mmlu_prox/sr/utils.py' 2026-05-11T14:30:17,611 adding 'lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_lite_sw.yaml' 2026-05-11T14:30:17,613 adding 'lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_sw.yaml' 2026-05-11T14:30:17,614 adding 'lm_eval/tasks/mmlu_prox/sw/_sw_lite_template_yaml' 2026-05-11T14:30:17,615 adding 'lm_eval/tasks/mmlu_prox/sw/_sw_template_yaml' 2026-05-11T14:30:17,616 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_biology.yaml' 2026-05-11T14:30:17,618 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_business.yaml' 2026-05-11T14:30:17,619 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_chemistry.yaml' 2026-05-11T14:30:17,620 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_computer_science.yaml' 2026-05-11T14:30:17,621 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_economics.yaml' 2026-05-11T14:30:17,622 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_engineering.yaml' 2026-05-11T14:30:17,624 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_health.yaml' 2026-05-11T14:30:17,625 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_history.yaml' 2026-05-11T14:30:17,626 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_law.yaml' 2026-05-11T14:30:17,627 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_math.yaml' 2026-05-11T14:30:17,628 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_other.yaml' 2026-05-11T14:30:17,629 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_philosophy.yaml' 2026-05-11T14:30:17,631 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_physics.yaml' 2026-05-11T14:30:17,632 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_psychology.yaml' 2026-05-11T14:30:17,633 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_biology.yaml' 2026-05-11T14:30:17,634 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_business.yaml' 2026-05-11T14:30:17,635 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_chemistry.yaml' 2026-05-11T14:30:17,636 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_computer_science.yaml' 2026-05-11T14:30:17,637 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_economics.yaml' 2026-05-11T14:30:17,639 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_engineering.yaml' 2026-05-11T14:30:17,640 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_health.yaml' 2026-05-11T14:30:17,641 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_history.yaml' 2026-05-11T14:30:17,642 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_law.yaml' 2026-05-11T14:30:17,643 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_math.yaml' 2026-05-11T14:30:17,644 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_other.yaml' 2026-05-11T14:30:17,646 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_philosophy.yaml' 2026-05-11T14:30:17,647 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_physics.yaml' 2026-05-11T14:30:17,648 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_psychology.yaml' 2026-05-11T14:30:17,649 adding 'lm_eval/tasks/mmlu_prox/sw/utils.py' 2026-05-11T14:30:17,651 adding 'lm_eval/tasks/mmlu_prox/te/_mmlu_prox_lite_te.yaml' 2026-05-11T14:30:17,653 adding 'lm_eval/tasks/mmlu_prox/te/_mmlu_prox_te.yaml' 2026-05-11T14:30:17,654 adding 'lm_eval/tasks/mmlu_prox/te/_te_lite_template_yaml' 2026-05-11T14:30:17,655 adding 'lm_eval/tasks/mmlu_prox/te/_te_template_yaml' 2026-05-11T14:30:17,657 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_biology.yaml' 2026-05-11T14:30:17,658 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_business.yaml' 2026-05-11T14:30:17,659 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_chemistry.yaml' 2026-05-11T14:30:17,660 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_computer_science.yaml' 2026-05-11T14:30:17,662 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_economics.yaml' 2026-05-11T14:30:17,663 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_engineering.yaml' 2026-05-11T14:30:17,664 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_health.yaml' 2026-05-11T14:30:17,665 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_history.yaml' 2026-05-11T14:30:17,666 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_law.yaml' 2026-05-11T14:30:17,668 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_math.yaml' 2026-05-11T14:30:17,669 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_other.yaml' 2026-05-11T14:30:17,670 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_philosophy.yaml' 2026-05-11T14:30:17,671 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_physics.yaml' 2026-05-11T14:30:17,672 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_psychology.yaml' 2026-05-11T14:30:17,673 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_biology.yaml' 2026-05-11T14:30:17,675 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_business.yaml' 2026-05-11T14:30:17,676 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_chemistry.yaml' 2026-05-11T14:30:17,677 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_computer_science.yaml' 2026-05-11T14:30:17,678 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_economics.yaml' 2026-05-11T14:30:17,679 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_engineering.yaml' 2026-05-11T14:30:17,681 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_health.yaml' 2026-05-11T14:30:17,682 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_history.yaml' 2026-05-11T14:30:17,683 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_law.yaml' 2026-05-11T14:30:17,684 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_math.yaml' 2026-05-11T14:30:17,685 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_other.yaml' 2026-05-11T14:30:17,686 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_philosophy.yaml' 2026-05-11T14:30:17,687 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_physics.yaml' 2026-05-11T14:30:17,688 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_psychology.yaml' 2026-05-11T14:30:17,689 adding 'lm_eval/tasks/mmlu_prox/te/utils.py' 2026-05-11T14:30:17,691 adding 'lm_eval/tasks/mmlu_prox/template/_lang_template_yaml' 2026-05-11T14:30:17,692 adding 'lm_eval/tasks/mmlu_prox/template/utils.py' 2026-05-11T14:30:17,694 adding 'lm_eval/tasks/mmlu_prox/th/_mmlu_prox_lite_th.yaml' 2026-05-11T14:30:17,696 adding 'lm_eval/tasks/mmlu_prox/th/_mmlu_prox_th.yaml' 2026-05-11T14:30:17,697 adding 'lm_eval/tasks/mmlu_prox/th/_th_lite_template_yaml' 2026-05-11T14:30:17,698 adding 'lm_eval/tasks/mmlu_prox/th/_th_template_yaml' 2026-05-11T14:30:17,699 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_biology.yaml' 2026-05-11T14:30:17,701 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_business.yaml' 2026-05-11T14:30:17,702 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_chemistry.yaml' 2026-05-11T14:30:17,703 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_computer_science.yaml' 2026-05-11T14:30:17,704 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_economics.yaml' 2026-05-11T14:30:17,705 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_engineering.yaml' 2026-05-11T14:30:17,706 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_health.yaml' 2026-05-11T14:30:17,708 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_history.yaml' 2026-05-11T14:30:17,709 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_law.yaml' 2026-05-11T14:30:17,710 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_math.yaml' 2026-05-11T14:30:17,711 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_other.yaml' 2026-05-11T14:30:17,713 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_philosophy.yaml' 2026-05-11T14:30:17,714 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_physics.yaml' 2026-05-11T14:30:17,715 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_psychology.yaml' 2026-05-11T14:30:17,716 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_biology.yaml' 2026-05-11T14:30:17,718 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_business.yaml' 2026-05-11T14:30:17,719 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_chemistry.yaml' 2026-05-11T14:30:17,720 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_computer_science.yaml' 2026-05-11T14:30:17,721 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_economics.yaml' 2026-05-11T14:30:17,722 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_engineering.yaml' 2026-05-11T14:30:17,724 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_health.yaml' 2026-05-11T14:30:17,725 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_history.yaml' 2026-05-11T14:30:17,726 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_law.yaml' 2026-05-11T14:30:17,727 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_math.yaml' 2026-05-11T14:30:17,729 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_other.yaml' 2026-05-11T14:30:17,730 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_philosophy.yaml' 2026-05-11T14:30:17,731 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_physics.yaml' 2026-05-11T14:30:17,732 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_psychology.yaml' 2026-05-11T14:30:17,734 adding 'lm_eval/tasks/mmlu_prox/th/utils.py' 2026-05-11T14:30:17,736 adding 'lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_lite_uk.yaml' 2026-05-11T14:30:17,737 adding 'lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_uk.yaml' 2026-05-11T14:30:17,738 adding 'lm_eval/tasks/mmlu_prox/uk/_uk_lite_template_yaml' 2026-05-11T14:30:17,739 adding 'lm_eval/tasks/mmlu_prox/uk/_uk_template_yaml' 2026-05-11T14:30:17,740 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_biology.yaml' 2026-05-11T14:30:17,741 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_business.yaml' 2026-05-11T14:30:17,742 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_chemistry.yaml' 2026-05-11T14:30:17,744 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_computer_science.yaml' 2026-05-11T14:30:17,745 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_economics.yaml' 2026-05-11T14:30:17,746 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_engineering.yaml' 2026-05-11T14:30:17,747 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_health.yaml' 2026-05-11T14:30:17,748 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_history.yaml' 2026-05-11T14:30:17,749 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_law.yaml' 2026-05-11T14:30:17,751 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_math.yaml' 2026-05-11T14:30:17,752 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_other.yaml' 2026-05-11T14:30:17,753 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_philosophy.yaml' 2026-05-11T14:30:17,754 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_physics.yaml' 2026-05-11T14:30:17,755 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_psychology.yaml' 2026-05-11T14:30:17,756 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_biology.yaml' 2026-05-11T14:30:17,758 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_business.yaml' 2026-05-11T14:30:17,759 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_chemistry.yaml' 2026-05-11T14:30:17,760 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_computer_science.yaml' 2026-05-11T14:30:17,761 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_economics.yaml' 2026-05-11T14:30:17,762 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_engineering.yaml' 2026-05-11T14:30:17,764 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_health.yaml' 2026-05-11T14:30:17,765 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_history.yaml' 2026-05-11T14:30:17,766 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_law.yaml' 2026-05-11T14:30:17,767 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_math.yaml' 2026-05-11T14:30:17,768 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_other.yaml' 2026-05-11T14:30:17,770 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_philosophy.yaml' 2026-05-11T14:30:17,771 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_physics.yaml' 2026-05-11T14:30:17,772 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_psychology.yaml' 2026-05-11T14:30:17,773 adding 'lm_eval/tasks/mmlu_prox/uk/utils.py' 2026-05-11T14:30:17,775 adding 'lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_lite_ur.yaml' 2026-05-11T14:30:17,777 adding 'lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_ur.yaml' 2026-05-11T14:30:17,778 adding 'lm_eval/tasks/mmlu_prox/ur/_ur_lite_template_yaml' 2026-05-11T14:30:17,779 adding 'lm_eval/tasks/mmlu_prox/ur/_ur_template_yaml' 2026-05-11T14:30:17,780 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_biology.yaml' 2026-05-11T14:30:17,781 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_business.yaml' 2026-05-11T14:30:17,783 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_chemistry.yaml' 2026-05-11T14:30:17,784 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_computer_science.yaml' 2026-05-11T14:30:17,785 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_economics.yaml' 2026-05-11T14:30:17,786 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_engineering.yaml' 2026-05-11T14:30:17,787 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_health.yaml' 2026-05-11T14:30:17,789 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_history.yaml' 2026-05-11T14:30:17,790 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_law.yaml' 2026-05-11T14:30:17,791 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_math.yaml' 2026-05-11T14:30:17,792 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_other.yaml' 2026-05-11T14:30:17,793 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_philosophy.yaml' 2026-05-11T14:30:17,794 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_physics.yaml' 2026-05-11T14:30:17,795 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_psychology.yaml' 2026-05-11T14:30:17,797 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_biology.yaml' 2026-05-11T14:30:17,798 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_business.yaml' 2026-05-11T14:30:17,799 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_chemistry.yaml' 2026-05-11T14:30:17,800 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_computer_science.yaml' 2026-05-11T14:30:17,801 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_economics.yaml' 2026-05-11T14:30:17,802 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_engineering.yaml' 2026-05-11T14:30:17,803 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_health.yaml' 2026-05-11T14:30:17,804 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_history.yaml' 2026-05-11T14:30:17,806 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_law.yaml' 2026-05-11T14:30:17,807 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_math.yaml' 2026-05-11T14:30:17,808 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_other.yaml' 2026-05-11T14:30:17,809 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_philosophy.yaml' 2026-05-11T14:30:17,810 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_physics.yaml' 2026-05-11T14:30:17,811 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_psychology.yaml' 2026-05-11T14:30:17,813 adding 'lm_eval/tasks/mmlu_prox/ur/utils.py' 2026-05-11T14:30:17,815 adding 'lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_lite_vi.yaml' 2026-05-11T14:30:17,816 adding 'lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_vi.yaml' 2026-05-11T14:30:17,817 adding 'lm_eval/tasks/mmlu_prox/vi/_vi_lite_template_yaml' 2026-05-11T14:30:17,819 adding 'lm_eval/tasks/mmlu_prox/vi/_vi_template_yaml' 2026-05-11T14:30:17,820 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_biology.yaml' 2026-05-11T14:30:17,821 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_business.yaml' 2026-05-11T14:30:17,822 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_chemistry.yaml' 2026-05-11T14:30:17,823 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_computer_science.yaml' 2026-05-11T14:30:17,825 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_economics.yaml' 2026-05-11T14:30:17,826 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_engineering.yaml' 2026-05-11T14:30:17,827 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_health.yaml' 2026-05-11T14:30:17,828 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_history.yaml' 2026-05-11T14:30:17,829 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_law.yaml' 2026-05-11T14:30:17,831 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_math.yaml' 2026-05-11T14:30:17,832 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_other.yaml' 2026-05-11T14:30:17,833 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_philosophy.yaml' 2026-05-11T14:30:17,834 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_physics.yaml' 2026-05-11T14:30:17,835 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_psychology.yaml' 2026-05-11T14:30:17,837 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_biology.yaml' 2026-05-11T14:30:17,838 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_business.yaml' 2026-05-11T14:30:17,839 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_chemistry.yaml' 2026-05-11T14:30:17,840 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_computer_science.yaml' 2026-05-11T14:30:17,842 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_economics.yaml' 2026-05-11T14:30:17,843 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_engineering.yaml' 2026-05-11T14:30:17,844 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_health.yaml' 2026-05-11T14:30:17,845 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_history.yaml' 2026-05-11T14:30:17,847 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_law.yaml' 2026-05-11T14:30:17,848 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_math.yaml' 2026-05-11T14:30:17,849 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_other.yaml' 2026-05-11T14:30:17,850 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_philosophy.yaml' 2026-05-11T14:30:17,851 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_physics.yaml' 2026-05-11T14:30:17,852 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_psychology.yaml' 2026-05-11T14:30:17,853 adding 'lm_eval/tasks/mmlu_prox/vi/utils.py' 2026-05-11T14:30:17,855 adding 'lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_lite_wo.yaml' 2026-05-11T14:30:17,856 adding 'lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_wo.yaml' 2026-05-11T14:30:17,857 adding 'lm_eval/tasks/mmlu_prox/wo/_wo_lite_template_yaml' 2026-05-11T14:30:17,858 adding 'lm_eval/tasks/mmlu_prox/wo/_wo_template_yaml' 2026-05-11T14:30:17,860 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_biology.yaml' 2026-05-11T14:30:17,861 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_business.yaml' 2026-05-11T14:30:17,862 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_chemistry.yaml' 2026-05-11T14:30:17,863 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_computer_science.yaml' 2026-05-11T14:30:17,864 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_economics.yaml' 2026-05-11T14:30:17,865 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_engineering.yaml' 2026-05-11T14:30:17,866 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_health.yaml' 2026-05-11T14:30:17,868 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_history.yaml' 2026-05-11T14:30:17,869 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_law.yaml' 2026-05-11T14:30:17,870 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_math.yaml' 2026-05-11T14:30:17,871 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_other.yaml' 2026-05-11T14:30:17,872 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_philosophy.yaml' 2026-05-11T14:30:17,874 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_physics.yaml' 2026-05-11T14:30:17,875 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_psychology.yaml' 2026-05-11T14:30:17,876 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_biology.yaml' 2026-05-11T14:30:17,877 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_business.yaml' 2026-05-11T14:30:17,878 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_chemistry.yaml' 2026-05-11T14:30:17,880 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_computer_science.yaml' 2026-05-11T14:30:17,881 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_economics.yaml' 2026-05-11T14:30:17,882 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_engineering.yaml' 2026-05-11T14:30:17,883 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_health.yaml' 2026-05-11T14:30:17,884 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_history.yaml' 2026-05-11T14:30:17,885 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_law.yaml' 2026-05-11T14:30:17,887 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_math.yaml' 2026-05-11T14:30:17,888 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_other.yaml' 2026-05-11T14:30:17,889 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_philosophy.yaml' 2026-05-11T14:30:17,890 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_physics.yaml' 2026-05-11T14:30:17,891 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_psychology.yaml' 2026-05-11T14:30:17,893 adding 'lm_eval/tasks/mmlu_prox/wo/utils.py' 2026-05-11T14:30:17,895 adding 'lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_lite_yo.yaml' 2026-05-11T14:30:17,896 adding 'lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_yo.yaml' 2026-05-11T14:30:17,897 adding 'lm_eval/tasks/mmlu_prox/yo/_yo_lite_template_yaml' 2026-05-11T14:30:17,899 adding 'lm_eval/tasks/mmlu_prox/yo/_yo_template_yaml' 2026-05-11T14:30:17,900 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_biology.yaml' 2026-05-11T14:30:17,901 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_business.yaml' 2026-05-11T14:30:17,902 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_chemistry.yaml' 2026-05-11T14:30:17,904 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_computer_science.yaml' 2026-05-11T14:30:17,905 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_economics.yaml' 2026-05-11T14:30:17,906 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_engineering.yaml' 2026-05-11T14:30:17,907 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_health.yaml' 2026-05-11T14:30:17,908 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_history.yaml' 2026-05-11T14:30:17,909 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_law.yaml' 2026-05-11T14:30:17,910 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_math.yaml' 2026-05-11T14:30:17,911 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_other.yaml' 2026-05-11T14:30:17,912 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_philosophy.yaml' 2026-05-11T14:30:17,913 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_physics.yaml' 2026-05-11T14:30:17,915 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_psychology.yaml' 2026-05-11T14:30:17,916 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_biology.yaml' 2026-05-11T14:30:17,917 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_business.yaml' 2026-05-11T14:30:17,918 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_chemistry.yaml' 2026-05-11T14:30:17,919 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_computer_science.yaml' 2026-05-11T14:30:17,920 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_economics.yaml' 2026-05-11T14:30:17,922 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_engineering.yaml' 2026-05-11T14:30:17,923 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_health.yaml' 2026-05-11T14:30:17,924 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_history.yaml' 2026-05-11T14:30:17,925 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_law.yaml' 2026-05-11T14:30:17,926 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_math.yaml' 2026-05-11T14:30:17,928 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_other.yaml' 2026-05-11T14:30:17,929 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_philosophy.yaml' 2026-05-11T14:30:17,930 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_physics.yaml' 2026-05-11T14:30:17,931 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_psychology.yaml' 2026-05-11T14:30:17,932 adding 'lm_eval/tasks/mmlu_prox/yo/utils.py' 2026-05-11T14:30:17,934 adding 'lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_lite_zh.yaml' 2026-05-11T14:30:17,936 adding 'lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_zh.yaml' 2026-05-11T14:30:17,937 adding 'lm_eval/tasks/mmlu_prox/zh/_zh_lite_template_yaml' 2026-05-11T14:30:17,938 adding 'lm_eval/tasks/mmlu_prox/zh/_zh_template_yaml' 2026-05-11T14:30:17,939 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_biology.yaml' 2026-05-11T14:30:17,941 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_business.yaml' 2026-05-11T14:30:17,942 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_chemistry.yaml' 2026-05-11T14:30:17,943 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_computer_science.yaml' 2026-05-11T14:30:17,944 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_economics.yaml' 2026-05-11T14:30:17,945 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_engineering.yaml' 2026-05-11T14:30:17,947 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_health.yaml' 2026-05-11T14:30:17,948 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_history.yaml' 2026-05-11T14:30:17,949 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_law.yaml' 2026-05-11T14:30:17,950 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_math.yaml' 2026-05-11T14:30:17,951 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_other.yaml' 2026-05-11T14:30:17,952 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_philosophy.yaml' 2026-05-11T14:30:17,954 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_physics.yaml' 2026-05-11T14:30:17,955 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_psychology.yaml' 2026-05-11T14:30:17,956 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_biology.yaml' 2026-05-11T14:30:17,957 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_business.yaml' 2026-05-11T14:30:17,958 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_chemistry.yaml' 2026-05-11T14:30:17,959 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_computer_science.yaml' 2026-05-11T14:30:17,960 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_economics.yaml' 2026-05-11T14:30:17,961 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_engineering.yaml' 2026-05-11T14:30:17,963 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_health.yaml' 2026-05-11T14:30:17,964 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_history.yaml' 2026-05-11T14:30:17,965 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_law.yaml' 2026-05-11T14:30:17,966 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_math.yaml' 2026-05-11T14:30:17,967 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_other.yaml' 2026-05-11T14:30:17,968 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_philosophy.yaml' 2026-05-11T14:30:17,969 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_physics.yaml' 2026-05-11T14:30:17,971 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_psychology.yaml' 2026-05-11T14:30:17,972 adding 'lm_eval/tasks/mmlu_prox/zh/utils.py' 2026-05-11T14:30:17,974 adding 'lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_lite_zu.yaml' 2026-05-11T14:30:17,975 adding 'lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_zu.yaml' 2026-05-11T14:30:17,976 adding 'lm_eval/tasks/mmlu_prox/zu/_zu_lite_template_yaml' 2026-05-11T14:30:17,977 adding 'lm_eval/tasks/mmlu_prox/zu/_zu_template_yaml' 2026-05-11T14:30:17,979 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_biology.yaml' 2026-05-11T14:30:17,980 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_business.yaml' 2026-05-11T14:30:17,981 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_chemistry.yaml' 2026-05-11T14:30:17,982 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_computer_science.yaml' 2026-05-11T14:30:17,983 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_economics.yaml' 2026-05-11T14:30:17,984 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_engineering.yaml' 2026-05-11T14:30:17,986 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_health.yaml' 2026-05-11T14:30:17,987 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_history.yaml' 2026-05-11T14:30:17,988 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_law.yaml' 2026-05-11T14:30:17,989 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_math.yaml' 2026-05-11T14:30:17,990 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_other.yaml' 2026-05-11T14:30:17,991 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_philosophy.yaml' 2026-05-11T14:30:17,993 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_physics.yaml' 2026-05-11T14:30:17,994 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_psychology.yaml' 2026-05-11T14:30:17,995 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_biology.yaml' 2026-05-11T14:30:17,996 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_business.yaml' 2026-05-11T14:30:17,997 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_chemistry.yaml' 2026-05-11T14:30:17,998 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_computer_science.yaml' 2026-05-11T14:30:18,000 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_economics.yaml' 2026-05-11T14:30:18,001 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_engineering.yaml' 2026-05-11T14:30:18,002 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_health.yaml' 2026-05-11T14:30:18,003 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_history.yaml' 2026-05-11T14:30:18,004 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_law.yaml' 2026-05-11T14:30:18,006 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_math.yaml' 2026-05-11T14:30:18,007 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_other.yaml' 2026-05-11T14:30:18,008 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_philosophy.yaml' 2026-05-11T14:30:18,009 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_physics.yaml' 2026-05-11T14:30:18,010 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_psychology.yaml' 2026-05-11T14:30:18,012 adding 'lm_eval/tasks/mmlu_prox/zu/utils.py' 2026-05-11T14:30:18,014 adding 'lm_eval/tasks/mmlusr/README.md' 2026-05-11T14:30:18,015 adding 'lm_eval/tasks/mmlusr/config.py' 2026-05-11T14:30:18,017 adding 'lm_eval/tasks/mmlusr/answer_only/_answer_only.yaml' 2026-05-11T14:30:18,019 adding 'lm_eval/tasks/mmlusr/answer_only/_mmlusr_a_yml' 2026-05-11T14:30:18,020 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_abstract_algebra.yaml' 2026-05-11T14:30:18,021 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_anatomy.yaml' 2026-05-11T14:30:18,022 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_astronomy.yaml' 2026-05-11T14:30:18,023 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_business_ethics.yaml' 2026-05-11T14:30:18,024 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_clinical_knowledge.yaml' 2026-05-11T14:30:18,025 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_biology.yaml' 2026-05-11T14:30:18,026 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_chemistry.yaml' 2026-05-11T14:30:18,028 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_computer_science.yaml' 2026-05-11T14:30:18,029 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_mathematics.yaml' 2026-05-11T14:30:18,030 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_medicine.yaml' 2026-05-11T14:30:18,031 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_physics.yaml' 2026-05-11T14:30:18,032 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_computer_security.yaml' 2026-05-11T14:30:18,033 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_conceptual_physics.yaml' 2026-05-11T14:30:18,034 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_econometrics.yaml' 2026-05-11T14:30:18,036 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_electrical_engineering.yaml' 2026-05-11T14:30:18,037 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_elementary_mathematics.yaml' 2026-05-11T14:30:18,038 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_formal_logic.yaml' 2026-05-11T14:30:18,039 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_global_facts.yaml' 2026-05-11T14:30:18,040 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_biology.yaml' 2026-05-11T14:30:18,041 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_chemistry.yaml' 2026-05-11T14:30:18,042 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_computer_science.yaml' 2026-05-11T14:30:18,044 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_european_history.yaml' 2026-05-11T14:30:18,045 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_geography.yaml' 2026-05-11T14:30:18,046 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_government_and_politics.yaml' 2026-05-11T14:30:18,047 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_macroeconomics.yaml' 2026-05-11T14:30:18,048 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_mathematics.yaml' 2026-05-11T14:30:18,050 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_microeconomics.yaml' 2026-05-11T14:30:18,051 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_physics.yaml' 2026-05-11T14:30:18,052 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_psychology.yaml' 2026-05-11T14:30:18,053 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_statistics.yaml' 2026-05-11T14:30:18,054 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_us_history.yaml' 2026-05-11T14:30:18,055 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_world_history.yaml' 2026-05-11T14:30:18,057 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_human_aging.yaml' 2026-05-11T14:30:18,058 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_human_sexuality.yaml' 2026-05-11T14:30:18,059 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_international_law.yaml' 2026-05-11T14:30:18,060 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_jurisprudence.yaml' 2026-05-11T14:30:18,061 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_logical_fallacies.yaml' 2026-05-11T14:30:18,062 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_machine_learning.yaml' 2026-05-11T14:30:18,063 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_management.yaml' 2026-05-11T14:30:18,065 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_marketing.yaml' 2026-05-11T14:30:18,066 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_medical_genetics.yaml' 2026-05-11T14:30:18,067 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_miscellaneous.yaml' 2026-05-11T14:30:18,068 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_moral_disputes.yaml' 2026-05-11T14:30:18,069 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_moral_scenarios.yaml' 2026-05-11T14:30:18,070 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_nutrition.yaml' 2026-05-11T14:30:18,071 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_philosophy.yaml' 2026-05-11T14:30:18,072 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_prehistory.yaml' 2026-05-11T14:30:18,074 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_accounting.yaml' 2026-05-11T14:30:18,075 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_law.yaml' 2026-05-11T14:30:18,076 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_medicine.yaml' 2026-05-11T14:30:18,077 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_psychology.yaml' 2026-05-11T14:30:18,078 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_public_relations.yaml' 2026-05-11T14:30:18,079 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_security_studies.yaml' 2026-05-11T14:30:18,080 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_sociology.yaml' 2026-05-11T14:30:18,081 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_us_foreign_policy.yaml' 2026-05-11T14:30:18,082 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_virology.yaml' 2026-05-11T14:30:18,084 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_world_religions.yaml' 2026-05-11T14:30:18,085 adding 'lm_eval/tasks/mmlusr/answer_only/utils.py' 2026-05-11T14:30:18,088 adding 'lm_eval/tasks/mmlusr/question_and_answer/_mmlusr_qna_yml' 2026-05-11T14:30:18,089 adding 'lm_eval/tasks/mmlusr/question_and_answer/_question_and_answer.yaml' 2026-05-11T14:30:18,090 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_abstract_algebra.yaml' 2026-05-11T14:30:18,091 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_anatomy.yaml' 2026-05-11T14:30:18,092 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_astronomy.yaml' 2026-05-11T14:30:18,093 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_business_ethics.yaml' 2026-05-11T14:30:18,095 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_clinical_knowledge.yaml' 2026-05-11T14:30:18,096 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_biology.yaml' 2026-05-11T14:30:18,097 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_chemistry.yaml' 2026-05-11T14:30:18,098 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_computer_science.yaml' 2026-05-11T14:30:18,100 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_mathematics.yaml' 2026-05-11T14:30:18,101 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_medicine.yaml' 2026-05-11T14:30:18,102 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_physics.yaml' 2026-05-11T14:30:18,103 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_computer_security.yaml' 2026-05-11T14:30:18,104 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_conceptual_physics.yaml' 2026-05-11T14:30:18,106 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_econometrics.yaml' 2026-05-11T14:30:18,107 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_electrical_engineering.yaml' 2026-05-11T14:30:18,108 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_elementary_mathematics.yaml' 2026-05-11T14:30:18,109 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_formal_logic.yaml' 2026-05-11T14:30:18,110 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_global_facts.yaml' 2026-05-11T14:30:18,111 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_biology.yaml' 2026-05-11T14:30:18,112 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_chemistry.yaml' 2026-05-11T14:30:18,113 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_computer_science.yaml' 2026-05-11T14:30:18,114 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_european_history.yaml' 2026-05-11T14:30:18,116 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_geography.yaml' 2026-05-11T14:30:18,117 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_government_and_politics.yaml' 2026-05-11T14:30:18,118 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_macroeconomics.yaml' 2026-05-11T14:30:18,119 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_mathematics.yaml' 2026-05-11T14:30:18,120 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_microeconomics.yaml' 2026-05-11T14:30:18,121 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_physics.yaml' 2026-05-11T14:30:18,122 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_psychology.yaml' 2026-05-11T14:30:18,124 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_statistics.yaml' 2026-05-11T14:30:18,125 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_us_history.yaml' 2026-05-11T14:30:18,126 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_world_history.yaml' 2026-05-11T14:30:18,127 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_aging.yaml' 2026-05-11T14:30:18,128 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_sexuality.yaml' 2026-05-11T14:30:18,129 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_international_law.yaml' 2026-05-11T14:30:18,131 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_jurisprudence.yaml' 2026-05-11T14:30:18,132 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_logical_fallacies.yaml' 2026-05-11T14:30:18,133 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_machine_learning.yaml' 2026-05-11T14:30:18,134 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_management.yaml' 2026-05-11T14:30:18,135 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_marketing.yaml' 2026-05-11T14:30:18,136 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_medical_genetics.yaml' 2026-05-11T14:30:18,138 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_miscellaneous.yaml' 2026-05-11T14:30:18,139 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_disputes.yaml' 2026-05-11T14:30:18,140 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_scenarios.yaml' 2026-05-11T14:30:18,141 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_nutrition.yaml' 2026-05-11T14:30:18,142 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_philosophy.yaml' 2026-05-11T14:30:18,144 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_prehistory.yaml' 2026-05-11T14:30:18,145 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_accounting.yaml' 2026-05-11T14:30:18,146 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_law.yaml' 2026-05-11T14:30:18,147 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_medicine.yaml' 2026-05-11T14:30:18,149 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_psychology.yaml' 2026-05-11T14:30:18,150 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_public_relations.yaml' 2026-05-11T14:30:18,151 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_security_studies.yaml' 2026-05-11T14:30:18,152 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_sociology.yaml' 2026-05-11T14:30:18,153 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_us_foreign_policy.yaml' 2026-05-11T14:30:18,154 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml' 2026-05-11T14:30:18,155 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml' 2026-05-11T14:30:18,157 adding 'lm_eval/tasks/mmlusr/question_and_answer/utils.py' 2026-05-11T14:30:18,159 adding 'lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml' 2026-05-11T14:30:18,160 adding 'lm_eval/tasks/mmlusr/question_only/_question_only.yaml' 2026-05-11T14:30:18,161 adding 'lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml' 2026-05-11T14:30:18,162 adding 'lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml' 2026-05-11T14:30:18,164 adding 'lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml' 2026-05-11T14:30:18,165 adding 'lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml' 2026-05-11T14:30:18,166 adding 'lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml' 2026-05-11T14:30:18,167 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml' 2026-05-11T14:30:18,168 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml' 2026-05-11T14:30:18,169 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml' 2026-05-11T14:30:18,171 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml' 2026-05-11T14:30:18,172 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml' 2026-05-11T14:30:18,173 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml' 2026-05-11T14:30:18,174 adding 'lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml' 2026-05-11T14:30:18,175 adding 'lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml' 2026-05-11T14:30:18,176 adding 'lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml' 2026-05-11T14:30:18,177 adding 'lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml' 2026-05-11T14:30:18,179 adding 'lm_eval/tasks/mmlusr/question_only/question_only_elementary_mathematics.yaml' 2026-05-11T14:30:18,180 adding 'lm_eval/tasks/mmlusr/question_only/question_only_formal_logic.yaml' 2026-05-11T14:30:18,181 adding 'lm_eval/tasks/mmlusr/question_only/question_only_global_facts.yaml' 2026-05-11T14:30:18,182 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_biology.yaml' 2026-05-11T14:30:18,183 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_chemistry.yaml' 2026-05-11T14:30:18,184 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_computer_science.yaml' 2026-05-11T14:30:18,186 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_european_history.yaml' 2026-05-11T14:30:18,187 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_geography.yaml' 2026-05-11T14:30:18,188 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_government_and_politics.yaml' 2026-05-11T14:30:18,189 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_macroeconomics.yaml' 2026-05-11T14:30:18,191 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_mathematics.yaml' 2026-05-11T14:30:18,192 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_microeconomics.yaml' 2026-05-11T14:30:18,193 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_physics.yaml' 2026-05-11T14:30:18,194 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_psychology.yaml' 2026-05-11T14:30:18,195 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_statistics.yaml' 2026-05-11T14:30:18,196 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_us_history.yaml' 2026-05-11T14:30:18,197 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_world_history.yaml' 2026-05-11T14:30:18,198 adding 'lm_eval/tasks/mmlusr/question_only/question_only_human_aging.yaml' 2026-05-11T14:30:18,199 adding 'lm_eval/tasks/mmlusr/question_only/question_only_human_sexuality.yaml' 2026-05-11T14:30:18,200 adding 'lm_eval/tasks/mmlusr/question_only/question_only_international_law.yaml' 2026-05-11T14:30:18,201 adding 'lm_eval/tasks/mmlusr/question_only/question_only_jurisprudence.yaml' 2026-05-11T14:30:18,203 adding 'lm_eval/tasks/mmlusr/question_only/question_only_logical_fallacies.yaml' 2026-05-11T14:30:18,204 adding 'lm_eval/tasks/mmlusr/question_only/question_only_machine_learning.yaml' 2026-05-11T14:30:18,205 adding 'lm_eval/tasks/mmlusr/question_only/question_only_management.yaml' 2026-05-11T14:30:18,206 adding 'lm_eval/tasks/mmlusr/question_only/question_only_marketing.yaml' 2026-05-11T14:30:18,207 adding 'lm_eval/tasks/mmlusr/question_only/question_only_medical_genetics.yaml' 2026-05-11T14:30:18,208 adding 'lm_eval/tasks/mmlusr/question_only/question_only_miscellaneous.yaml' 2026-05-11T14:30:18,209 adding 'lm_eval/tasks/mmlusr/question_only/question_only_moral_disputes.yaml' 2026-05-11T14:30:18,210 adding 'lm_eval/tasks/mmlusr/question_only/question_only_moral_scenarios.yaml' 2026-05-11T14:30:18,212 adding 'lm_eval/tasks/mmlusr/question_only/question_only_nutrition.yaml' 2026-05-11T14:30:18,213 adding 'lm_eval/tasks/mmlusr/question_only/question_only_philosophy.yaml' 2026-05-11T14:30:18,214 adding 'lm_eval/tasks/mmlusr/question_only/question_only_prehistory.yaml' 2026-05-11T14:30:18,215 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_accounting.yaml' 2026-05-11T14:30:18,216 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_law.yaml' 2026-05-11T14:30:18,217 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_medicine.yaml' 2026-05-11T14:30:18,219 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_psychology.yaml' 2026-05-11T14:30:18,220 adding 'lm_eval/tasks/mmlusr/question_only/question_only_public_relations.yaml' 2026-05-11T14:30:18,221 adding 'lm_eval/tasks/mmlusr/question_only/question_only_security_studies.yaml' 2026-05-11T14:30:18,222 adding 'lm_eval/tasks/mmlusr/question_only/question_only_sociology.yaml' 2026-05-11T14:30:18,223 adding 'lm_eval/tasks/mmlusr/question_only/question_only_us_foreign_policy.yaml' 2026-05-11T14:30:18,224 adding 'lm_eval/tasks/mmlusr/question_only/question_only_virology.yaml' 2026-05-11T14:30:18,225 adding 'lm_eval/tasks/mmlusr/question_only/question_only_world_religions.yaml' 2026-05-11T14:30:18,227 adding 'lm_eval/tasks/mmlusr/question_only/utils.py' 2026-05-11T14:30:18,229 adding 'lm_eval/tasks/mmmu/README.md' 2026-05-11T14:30:18,231 adding 'lm_eval/tasks/mmmu/_art_and_design.yaml' 2026-05-11T14:30:18,232 adding 'lm_eval/tasks/mmmu/_business.yaml' 2026-05-11T14:30:18,233 adding 'lm_eval/tasks/mmmu/_health_and_medicine.yaml' 2026-05-11T14:30:18,234 adding 'lm_eval/tasks/mmmu/_humanities_and_social_sciences.yaml' 2026-05-11T14:30:18,236 adding 'lm_eval/tasks/mmmu/_mmmu.yaml' 2026-05-11T14:30:18,237 adding 'lm_eval/tasks/mmmu/_science.yaml' 2026-05-11T14:30:18,238 adding 'lm_eval/tasks/mmmu/_tech_and_engineering.yaml' 2026-05-11T14:30:18,239 adding 'lm_eval/tasks/mmmu/_template_yaml' 2026-05-11T14:30:18,240 adding 'lm_eval/tasks/mmmu/mmmu_accounting.yaml' 2026-05-11T14:30:18,241 adding 'lm_eval/tasks/mmmu/mmmu_agriculture.yaml' 2026-05-11T14:30:18,242 adding 'lm_eval/tasks/mmmu/mmmu_architecture_and_engineering.yaml' 2026-05-11T14:30:18,243 adding 'lm_eval/tasks/mmmu/mmmu_art.yaml' 2026-05-11T14:30:18,244 adding 'lm_eval/tasks/mmmu/mmmu_art_theory.yaml' 2026-05-11T14:30:18,245 adding 'lm_eval/tasks/mmmu/mmmu_basic_medical_science.yaml' 2026-05-11T14:30:18,246 adding 'lm_eval/tasks/mmmu/mmmu_biology.yaml' 2026-05-11T14:30:18,247 adding 'lm_eval/tasks/mmmu/mmmu_chemistry.yaml' 2026-05-11T14:30:18,248 adding 'lm_eval/tasks/mmmu/mmmu_clinical_medicine.yaml' 2026-05-11T14:30:18,250 adding 'lm_eval/tasks/mmmu/mmmu_computer_science.yaml' 2026-05-11T14:30:18,251 adding 'lm_eval/tasks/mmmu/mmmu_design.yaml' 2026-05-11T14:30:18,252 adding 'lm_eval/tasks/mmmu/mmmu_diagnostics_and_laboratory_medicine.yaml' 2026-05-11T14:30:18,253 adding 'lm_eval/tasks/mmmu/mmmu_economics.yaml' 2026-05-11T14:30:18,254 adding 'lm_eval/tasks/mmmu/mmmu_electronics.yaml' 2026-05-11T14:30:18,255 adding 'lm_eval/tasks/mmmu/mmmu_energy_and_power.yaml' 2026-05-11T14:30:18,256 adding 'lm_eval/tasks/mmmu/mmmu_finance.yaml' 2026-05-11T14:30:18,257 adding 'lm_eval/tasks/mmmu/mmmu_geography.yaml' 2026-05-11T14:30:18,258 adding 'lm_eval/tasks/mmmu/mmmu_history.yaml' 2026-05-11T14:30:18,260 adding 'lm_eval/tasks/mmmu/mmmu_literature.yaml' 2026-05-11T14:30:18,261 adding 'lm_eval/tasks/mmmu/mmmu_manage.yaml' 2026-05-11T14:30:18,262 adding 'lm_eval/tasks/mmmu/mmmu_marketing.yaml' 2026-05-11T14:30:18,263 adding 'lm_eval/tasks/mmmu/mmmu_materials.yaml' 2026-05-11T14:30:18,264 adding 'lm_eval/tasks/mmmu/mmmu_math.yaml' 2026-05-11T14:30:18,265 adding 'lm_eval/tasks/mmmu/mmmu_mechanical_engineering.yaml' 2026-05-11T14:30:18,266 adding 'lm_eval/tasks/mmmu/mmmu_music.yaml' 2026-05-11T14:30:18,267 adding 'lm_eval/tasks/mmmu/mmmu_pharmacy.yaml' 2026-05-11T14:30:18,268 adding 'lm_eval/tasks/mmmu/mmmu_physics.yaml' 2026-05-11T14:30:18,270 adding 'lm_eval/tasks/mmmu/mmmu_psychology.yaml' 2026-05-11T14:30:18,271 adding 'lm_eval/tasks/mmmu/mmmu_public_health.yaml' 2026-05-11T14:30:18,272 adding 'lm_eval/tasks/mmmu/mmmu_sociology.yaml' 2026-05-11T14:30:18,274 adding 'lm_eval/tasks/mmmu/utils.py' 2026-05-11T14:30:18,277 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py' 2026-05-11T14:30:18,278 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml' 2026-05-11T14:30:18,279 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml' 2026-05-11T14:30:18,280 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml' 2026-05-11T14:30:18,282 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml' 2026-05-11T14:30:18,283 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml' 2026-05-11T14:30:18,284 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml' 2026-05-11T14:30:18,285 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml' 2026-05-11T14:30:18,286 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml' 2026-05-11T14:30:18,287 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml' 2026-05-11T14:30:18,289 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml' 2026-05-11T14:30:18,290 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml' 2026-05-11T14:30:18,291 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml' 2026-05-11T14:30:18,292 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml' 2026-05-11T14:30:18,293 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml' 2026-05-11T14:30:18,294 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml' 2026-05-11T14:30:18,295 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml' 2026-05-11T14:30:18,296 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml' 2026-05-11T14:30:18,298 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml' 2026-05-11T14:30:18,299 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml' 2026-05-11T14:30:18,300 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml' 2026-05-11T14:30:18,301 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml' 2026-05-11T14:30:18,302 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml' 2026-05-11T14:30:18,303 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml' 2026-05-11T14:30:18,304 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml' 2026-05-11T14:30:18,305 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml' 2026-05-11T14:30:18,306 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml' 2026-05-11T14:30:18,307 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml' 2026-05-11T14:30:18,308 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml' 2026-05-11T14:30:18,310 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml' 2026-05-11T14:30:18,311 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml' 2026-05-11T14:30:18,312 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml' 2026-05-11T14:30:18,313 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml' 2026-05-11T14:30:18,314 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml' 2026-05-11T14:30:18,315 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml' 2026-05-11T14:30:18,316 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml' 2026-05-11T14:30:18,317 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml' 2026-05-11T14:30:18,318 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml' 2026-05-11T14:30:18,320 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml' 2026-05-11T14:30:18,321 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml' 2026-05-11T14:30:18,322 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml' 2026-05-11T14:30:18,323 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml' 2026-05-11T14:30:18,324 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml' 2026-05-11T14:30:18,325 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml' 2026-05-11T14:30:18,326 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml' 2026-05-11T14:30:18,328 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml' 2026-05-11T14:30:18,329 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml' 2026-05-11T14:30:18,330 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml' 2026-05-11T14:30:18,331 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml' 2026-05-11T14:30:18,332 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml' 2026-05-11T14:30:18,333 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml' 2026-05-11T14:30:18,337 adding 'lm_eval/tasks/model_written_evals/persona/_generate_configs.py' 2026-05-11T14:30:18,338 adding 'lm_eval/tasks/model_written_evals/persona/_template_yaml' 2026-05-11T14:30:18,339 adding 'lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml' 2026-05-11T14:30:18,340 adding 'lm_eval/tasks/model_written_evals/persona/agreeableness.yaml' 2026-05-11T14:30:18,341 adding 'lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml' 2026-05-11T14:30:18,343 adding 'lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml' 2026-05-11T14:30:18,344 adding 'lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml' 2026-05-11T14:30:18,345 adding 'lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml' 2026-05-11T14:30:18,346 adding 'lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml' 2026-05-11T14:30:18,347 adding 'lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml' 2026-05-11T14:30:18,348 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml' 2026-05-11T14:30:18,349 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml' 2026-05-11T14:30:18,351 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml' 2026-05-11T14:30:18,352 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml' 2026-05-11T14:30:18,353 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml' 2026-05-11T14:30:18,354 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml' 2026-05-11T14:30:18,355 adding 'lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml' 2026-05-11T14:30:18,356 adding 'lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml' 2026-05-11T14:30:18,357 adding 'lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml' 2026-05-11T14:30:18,359 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml' 2026-05-11T14:30:18,360 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml' 2026-05-11T14:30:18,361 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml' 2026-05-11T14:30:18,362 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml' 2026-05-11T14:30:18,363 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml' 2026-05-11T14:30:18,364 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml' 2026-05-11T14:30:18,366 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml' 2026-05-11T14:30:18,367 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml' 2026-05-11T14:30:18,368 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml' 2026-05-11T14:30:18,369 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml' 2026-05-11T14:30:18,370 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml' 2026-05-11T14:30:18,371 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml' 2026-05-11T14:30:18,372 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml' 2026-05-11T14:30:18,374 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml' 2026-05-11T14:30:18,375 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml' 2026-05-11T14:30:18,376 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml' 2026-05-11T14:30:18,377 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml' 2026-05-11T14:30:18,378 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml' 2026-05-11T14:30:18,379 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml' 2026-05-11T14:30:18,380 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml' 2026-05-11T14:30:18,381 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml' 2026-05-11T14:30:18,382 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml' 2026-05-11T14:30:18,383 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml' 2026-05-11T14:30:18,384 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml' 2026-05-11T14:30:18,385 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml' 2026-05-11T14:30:18,387 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml' 2026-05-11T14:30:18,388 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml' 2026-05-11T14:30:18,389 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml' 2026-05-11T14:30:18,390 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml' 2026-05-11T14:30:18,392 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml' 2026-05-11T14:30:18,393 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml' 2026-05-11T14:30:18,394 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml' 2026-05-11T14:30:18,395 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml' 2026-05-11T14:30:18,396 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml' 2026-05-11T14:30:18,397 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml' 2026-05-11T14:30:18,398 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml' 2026-05-11T14:30:18,400 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml' 2026-05-11T14:30:18,401 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml' 2026-05-11T14:30:18,402 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml' 2026-05-11T14:30:18,403 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml' 2026-05-11T14:30:18,404 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml' 2026-05-11T14:30:18,405 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml' 2026-05-11T14:30:18,406 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml' 2026-05-11T14:30:18,408 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml' 2026-05-11T14:30:18,409 adding 'lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml' 2026-05-11T14:30:18,410 adding 'lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml' 2026-05-11T14:30:18,411 adding 'lm_eval/tasks/model_written_evals/persona/extraversion.yaml' 2026-05-11T14:30:18,412 adding 'lm_eval/tasks/model_written_evals/persona/has-disability.yaml' 2026-05-11T14:30:18,413 adding 'lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml' 2026-05-11T14:30:18,414 adding 'lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml' 2026-05-11T14:30:18,415 adding 'lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml' 2026-05-11T14:30:18,416 adding 'lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml' 2026-05-11T14:30:18,418 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml' 2026-05-11T14:30:18,419 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml' 2026-05-11T14:30:18,420 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml' 2026-05-11T14:30:18,421 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml' 2026-05-11T14:30:18,422 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml' 2026-05-11T14:30:18,423 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml' 2026-05-11T14:30:18,425 adding 'lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml' 2026-05-11T14:30:18,426 adding 'lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml' 2026-05-11T14:30:18,427 adding 'lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml' 2026-05-11T14:30:18,428 adding 'lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml' 2026-05-11T14:30:18,429 adding 'lm_eval/tasks/model_written_evals/persona/narcissism.yaml' 2026-05-11T14:30:18,430 adding 'lm_eval/tasks/model_written_evals/persona/neuroticism.yaml' 2026-05-11T14:30:18,431 adding 'lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml' 2026-05-11T14:30:18,432 adding 'lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml' 2026-05-11T14:30:18,434 adding 'lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml' 2026-05-11T14:30:18,435 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml' 2026-05-11T14:30:18,436 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml' 2026-05-11T14:30:18,437 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml' 2026-05-11T14:30:18,438 adding 'lm_eval/tasks/model_written_evals/persona/openness.yaml' 2026-05-11T14:30:18,439 adding 'lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml' 2026-05-11T14:30:18,440 adding 'lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml' 2026-05-11T14:30:18,442 adding 'lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml' 2026-05-11T14:30:18,443 adding 'lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml' 2026-05-11T14:30:18,444 adding 'lm_eval/tasks/model_written_evals/persona/psychopathy.yaml' 2026-05-11T14:30:18,445 adding 'lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml' 2026-05-11T14:30:18,446 adding 'lm_eval/tasks/model_written_evals/persona/risk-averse.yaml' 2026-05-11T14:30:18,447 adding 'lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml' 2026-05-11T14:30:18,448 adding 'lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml' 2026-05-11T14:30:18,450 adding 'lm_eval/tasks/model_written_evals/persona/self-replication.yaml' 2026-05-11T14:30:18,451 adding 'lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml' 2026-05-11T14:30:18,452 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml' 2026-05-11T14:30:18,453 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml' 2026-05-11T14:30:18,454 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml' 2026-05-11T14:30:18,455 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml' 2026-05-11T14:30:18,457 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml' 2026-05-11T14:30:18,458 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml' 2026-05-11T14:30:18,459 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml' 2026-05-11T14:30:18,460 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml' 2026-05-11T14:30:18,461 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml' 2026-05-11T14:30:18,462 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml' 2026-05-11T14:30:18,463 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml' 2026-05-11T14:30:18,464 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml' 2026-05-11T14:30:18,465 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml' 2026-05-11T14:30:18,466 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml' 2026-05-11T14:30:18,467 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml' 2026-05-11T14:30:18,468 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml' 2026-05-11T14:30:18,470 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml' 2026-05-11T14:30:18,471 adding 'lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml' 2026-05-11T14:30:18,472 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml' 2026-05-11T14:30:18,473 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml' 2026-05-11T14:30:18,474 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml' 2026-05-11T14:30:18,475 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml' 2026-05-11T14:30:18,476 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml' 2026-05-11T14:30:18,477 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml' 2026-05-11T14:30:18,478 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml' 2026-05-11T14:30:18,480 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml' 2026-05-11T14:30:18,481 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml' 2026-05-11T14:30:18,482 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml' 2026-05-11T14:30:18,483 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml' 2026-05-11T14:30:18,484 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml' 2026-05-11T14:30:18,485 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml' 2026-05-11T14:30:18,487 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml' 2026-05-11T14:30:18,488 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml' 2026-05-11T14:30:18,489 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml' 2026-05-11T14:30:18,490 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml' 2026-05-11T14:30:18,491 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml' 2026-05-11T14:30:18,493 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml' 2026-05-11T14:30:18,494 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml' 2026-05-11T14:30:18,495 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml' 2026-05-11T14:30:18,497 adding 'lm_eval/tasks/model_written_evals/winogenerated/winogenerated' 2026-05-11T14:30:18,499 adding 'lm_eval/tasks/moral_stories/README.md' 2026-05-11T14:30:18,500 adding 'lm_eval/tasks/moral_stories/moral_stories.yaml' 2026-05-11T14:30:18,502 adding 'lm_eval/tasks/moral_stories/utils.py' 2026-05-11T14:30:18,503 adding 'lm_eval/tasks/mts_dialog/README.md' 2026-05-11T14:30:18,505 adding 'lm_eval/tasks/mts_dialog/mts_dialog.yaml' 2026-05-11T14:30:18,506 adding 'lm_eval/tasks/mts_dialog/mts_dialog_perplexity.yaml' 2026-05-11T14:30:18,507 adding 'lm_eval/tasks/mts_dialog/utils.py' 2026-05-11T14:30:18,508 adding 'lm_eval/tasks/mts_dialog/utils_perplexity.py' 2026-05-11T14:30:18,512 adding 'lm_eval/tasks/multiblimp/README.md' 2026-05-11T14:30:18,514 adding 'lm_eval/tasks/multiblimp/_template_yaml' 2026-05-11T14:30:18,515 adding 'lm_eval/tasks/multiblimp/multiblimp_abk.yaml' 2026-05-11T14:30:18,516 adding 'lm_eval/tasks/multiblimp/multiblimp_aln.yaml' 2026-05-11T14:30:18,517 adding 'lm_eval/tasks/multiblimp/multiblimp_amh.yaml' 2026-05-11T14:30:18,518 adding 'lm_eval/tasks/multiblimp/multiblimp_apu.yaml' 2026-05-11T14:30:18,519 adding 'lm_eval/tasks/multiblimp/multiblimp_aqz.yaml' 2026-05-11T14:30:18,520 adding 'lm_eval/tasks/multiblimp/multiblimp_arb.yaml' 2026-05-11T14:30:18,521 adding 'lm_eval/tasks/multiblimp/multiblimp_azz.yaml' 2026-05-11T14:30:18,522 adding 'lm_eval/tasks/multiblimp/multiblimp_bel.yaml' 2026-05-11T14:30:18,523 adding 'lm_eval/tasks/multiblimp/multiblimp_ben.yaml' 2026-05-11T14:30:18,525 adding 'lm_eval/tasks/multiblimp/multiblimp_bho.yaml' 2026-05-11T14:30:18,526 adding 'lm_eval/tasks/multiblimp/multiblimp_bor.yaml' 2026-05-11T14:30:18,527 adding 'lm_eval/tasks/multiblimp/multiblimp_bre.yaml' 2026-05-11T14:30:18,528 adding 'lm_eval/tasks/multiblimp/multiblimp_bua.yaml' 2026-05-11T14:30:18,529 adding 'lm_eval/tasks/multiblimp/multiblimp_bul.yaml' 2026-05-11T14:30:18,530 adding 'lm_eval/tasks/multiblimp/multiblimp_cat.yaml' 2026-05-11T14:30:18,531 adding 'lm_eval/tasks/multiblimp/multiblimp_ces.yaml' 2026-05-11T14:30:18,532 adding 'lm_eval/tasks/multiblimp/multiblimp_chu.yaml' 2026-05-11T14:30:18,534 adding 'lm_eval/tasks/multiblimp/multiblimp_cym.yaml' 2026-05-11T14:30:18,535 adding 'lm_eval/tasks/multiblimp/multiblimp_dan.yaml' 2026-05-11T14:30:18,536 adding 'lm_eval/tasks/multiblimp/multiblimp_deu.yaml' 2026-05-11T14:30:18,537 adding 'lm_eval/tasks/multiblimp/multiblimp_egy.yaml' 2026-05-11T14:30:18,538 adding 'lm_eval/tasks/multiblimp/multiblimp_ell.yaml' 2026-05-11T14:30:18,539 adding 'lm_eval/tasks/multiblimp/multiblimp_eng.yaml' 2026-05-11T14:30:18,540 adding 'lm_eval/tasks/multiblimp/multiblimp_est.yaml' 2026-05-11T14:30:18,541 adding 'lm_eval/tasks/multiblimp/multiblimp_eus.yaml' 2026-05-11T14:30:18,543 adding 'lm_eval/tasks/multiblimp/multiblimp_fao.yaml' 2026-05-11T14:30:18,544 adding 'lm_eval/tasks/multiblimp/multiblimp_fas.yaml' 2026-05-11T14:30:18,545 adding 'lm_eval/tasks/multiblimp/multiblimp_fin.yaml' 2026-05-11T14:30:18,546 adding 'lm_eval/tasks/multiblimp/multiblimp_fra.yaml' 2026-05-11T14:30:18,547 adding 'lm_eval/tasks/multiblimp/multiblimp_frm.yaml' 2026-05-11T14:30:18,548 adding 'lm_eval/tasks/multiblimp/multiblimp_fro.yaml' 2026-05-11T14:30:18,549 adding 'lm_eval/tasks/multiblimp/multiblimp_gla.yaml' 2026-05-11T14:30:18,550 adding 'lm_eval/tasks/multiblimp/multiblimp_gle.yaml' 2026-05-11T14:30:18,552 adding 'lm_eval/tasks/multiblimp/multiblimp_glg.yaml' 2026-05-11T14:30:18,553 adding 'lm_eval/tasks/multiblimp/multiblimp_got.yaml' 2026-05-11T14:30:18,554 adding 'lm_eval/tasks/multiblimp/multiblimp_grc.yaml' 2026-05-11T14:30:18,555 adding 'lm_eval/tasks/multiblimp/multiblimp_guj.yaml' 2026-05-11T14:30:18,556 adding 'lm_eval/tasks/multiblimp/multiblimp_hbo.yaml' 2026-05-11T14:30:18,557 adding 'lm_eval/tasks/multiblimp/multiblimp_hbs.yaml' 2026-05-11T14:30:18,558 adding 'lm_eval/tasks/multiblimp/multiblimp_heb.yaml' 2026-05-11T14:30:18,559 adding 'lm_eval/tasks/multiblimp/multiblimp_hin.yaml' 2026-05-11T14:30:18,560 adding 'lm_eval/tasks/multiblimp/multiblimp_hit.yaml' 2026-05-11T14:30:18,561 adding 'lm_eval/tasks/multiblimp/multiblimp_hsb.yaml' 2026-05-11T14:30:18,562 adding 'lm_eval/tasks/multiblimp/multiblimp_hun.yaml' 2026-05-11T14:30:18,563 adding 'lm_eval/tasks/multiblimp/multiblimp_hye.yaml' 2026-05-11T14:30:18,564 adding 'lm_eval/tasks/multiblimp/multiblimp_hyw.yaml' 2026-05-11T14:30:18,565 adding 'lm_eval/tasks/multiblimp/multiblimp_isl.yaml' 2026-05-11T14:30:18,566 adding 'lm_eval/tasks/multiblimp/multiblimp_ita.yaml' 2026-05-11T14:30:18,567 adding 'lm_eval/tasks/multiblimp/multiblimp_kat.yaml' 2026-05-11T14:30:18,568 adding 'lm_eval/tasks/multiblimp/multiblimp_kaz.yaml' 2026-05-11T14:30:18,569 adding 'lm_eval/tasks/multiblimp/multiblimp_kir.yaml' 2026-05-11T14:30:18,570 adding 'lm_eval/tasks/multiblimp/multiblimp_kmr.yaml' 2026-05-11T14:30:18,571 adding 'lm_eval/tasks/multiblimp/multiblimp_koi.yaml' 2026-05-11T14:30:18,573 adding 'lm_eval/tasks/multiblimp/multiblimp_kpv.yaml' 2026-05-11T14:30:18,574 adding 'lm_eval/tasks/multiblimp/multiblimp_krl.yaml' 2026-05-11T14:30:18,575 adding 'lm_eval/tasks/multiblimp/multiblimp_kxh.yaml' 2026-05-11T14:30:18,576 adding 'lm_eval/tasks/multiblimp/multiblimp_lat.yaml' 2026-05-11T14:30:18,577 adding 'lm_eval/tasks/multiblimp/multiblimp_lav.yaml' 2026-05-11T14:30:18,578 adding 'lm_eval/tasks/multiblimp/multiblimp_lij.yaml' 2026-05-11T14:30:18,579 adding 'lm_eval/tasks/multiblimp/multiblimp_lit.yaml' 2026-05-11T14:30:18,580 adding 'lm_eval/tasks/multiblimp/multiblimp_mar.yaml' 2026-05-11T14:30:18,581 adding 'lm_eval/tasks/multiblimp/multiblimp_mdf.yaml' 2026-05-11T14:30:18,582 adding 'lm_eval/tasks/multiblimp/multiblimp_mkd.yaml' 2026-05-11T14:30:18,583 adding 'lm_eval/tasks/multiblimp/multiblimp_myv.yaml' 2026-05-11T14:30:18,584 adding 'lm_eval/tasks/multiblimp/multiblimp_nds.yaml' 2026-05-11T14:30:18,585 adding 'lm_eval/tasks/multiblimp/multiblimp_nhi.yaml' 2026-05-11T14:30:18,586 adding 'lm_eval/tasks/multiblimp/multiblimp_nld.yaml' 2026-05-11T14:30:18,588 adding 'lm_eval/tasks/multiblimp/multiblimp_olo.yaml' 2026-05-11T14:30:18,589 adding 'lm_eval/tasks/multiblimp/multiblimp_orv.yaml' 2026-05-11T14:30:18,590 adding 'lm_eval/tasks/multiblimp/multiblimp_ota.yaml' 2026-05-11T14:30:18,591 adding 'lm_eval/tasks/multiblimp/multiblimp_pcm.yaml' 2026-05-11T14:30:18,592 adding 'lm_eval/tasks/multiblimp/multiblimp_pol.yaml' 2026-05-11T14:30:18,593 adding 'lm_eval/tasks/multiblimp/multiblimp_por.yaml' 2026-05-11T14:30:18,594 adding 'lm_eval/tasks/multiblimp/multiblimp_quc.yaml' 2026-05-11T14:30:18,595 adding 'lm_eval/tasks/multiblimp/multiblimp_ron.yaml' 2026-05-11T14:30:18,596 adding 'lm_eval/tasks/multiblimp/multiblimp_rus.yaml' 2026-05-11T14:30:18,597 adding 'lm_eval/tasks/multiblimp/multiblimp_sah.yaml' 2026-05-11T14:30:18,599 adding 'lm_eval/tasks/multiblimp/multiblimp_san.yaml' 2026-05-11T14:30:18,600 adding 'lm_eval/tasks/multiblimp/multiblimp_slk.yaml' 2026-05-11T14:30:18,601 adding 'lm_eval/tasks/multiblimp/multiblimp_slv.yaml' 2026-05-11T14:30:18,602 adding 'lm_eval/tasks/multiblimp/multiblimp_sme.yaml' 2026-05-11T14:30:18,603 adding 'lm_eval/tasks/multiblimp/multiblimp_sms.yaml' 2026-05-11T14:30:18,604 adding 'lm_eval/tasks/multiblimp/multiblimp_spa.yaml' 2026-05-11T14:30:18,605 adding 'lm_eval/tasks/multiblimp/multiblimp_sqi.yaml' 2026-05-11T14:30:18,606 adding 'lm_eval/tasks/multiblimp/multiblimp_swe.yaml' 2026-05-11T14:30:18,608 adding 'lm_eval/tasks/multiblimp/multiblimp_tam.yaml' 2026-05-11T14:30:18,609 adding 'lm_eval/tasks/multiblimp/multiblimp_tpn.yaml' 2026-05-11T14:30:18,610 adding 'lm_eval/tasks/multiblimp/multiblimp_ttc.yaml' 2026-05-11T14:30:18,611 adding 'lm_eval/tasks/multiblimp/multiblimp_tur.yaml' 2026-05-11T14:30:18,612 adding 'lm_eval/tasks/multiblimp/multiblimp_uig.yaml' 2026-05-11T14:30:18,613 adding 'lm_eval/tasks/multiblimp/multiblimp_ukr.yaml' 2026-05-11T14:30:18,614 adding 'lm_eval/tasks/multiblimp/multiblimp_urb.yaml' 2026-05-11T14:30:18,615 adding 'lm_eval/tasks/multiblimp/multiblimp_urd.yaml' 2026-05-11T14:30:18,616 adding 'lm_eval/tasks/multiblimp/multiblimp_uzb.yaml' 2026-05-11T14:30:18,617 adding 'lm_eval/tasks/multiblimp/multiblimp_vep.yaml' 2026-05-11T14:30:18,618 adding 'lm_eval/tasks/multiblimp/multiblimp_wbp.yaml' 2026-05-11T14:30:18,619 adding 'lm_eval/tasks/multiblimp/multiblimp_wol.yaml' 2026-05-11T14:30:18,620 adding 'lm_eval/tasks/multiblimp/multiblimp_xcl.yaml' 2026-05-11T14:30:18,621 adding 'lm_eval/tasks/multiblimp/multiblimp_xnr.yaml' 2026-05-11T14:30:18,622 adding 'lm_eval/tasks/multiblimp/multiblimp_xpg.yaml' 2026-05-11T14:30:18,623 adding 'lm_eval/tasks/multiblimp/multiblimp_yrl.yaml' 2026-05-11T14:30:18,625 adding 'lm_eval/tasks/mutual/README.md' 2026-05-11T14:30:18,626 adding 'lm_eval/tasks/mutual/multual_plus.yaml' 2026-05-11T14:30:18,627 adding 'lm_eval/tasks/mutual/mutual.yaml' 2026-05-11T14:30:18,628 adding 'lm_eval/tasks/mutual/utils.py' 2026-05-11T14:30:18,631 adding 'lm_eval/tasks/nerel-bench/README.md' 2026-05-11T14:30:18,632 adding 'lm_eval/tasks/nerel-bench/_template_yaml' 2026-05-11T14:30:18,633 adding 'lm_eval/tasks/nerel-bench/ru_entity_definition.yaml' 2026-05-11T14:30:18,634 adding 'lm_eval/tasks/nerel-bench/ru_entity_recognition.yaml' 2026-05-11T14:30:18,635 adding 'lm_eval/tasks/nerel-bench/ru_relation_definition.yaml' 2026-05-11T14:30:18,636 adding 'lm_eval/tasks/nerel-bench/ru_relation_extraction.yaml' 2026-05-11T14:30:18,643 adding 'lm_eval/tasks/nerel-bench/utils.py' 2026-05-11T14:30:18,646 adding 'lm_eval/tasks/noreval/README.md' 2026-05-11T14:30:18,727 adding 'lm_eval/tasks/noreval/noreval.jpg' 2026-05-11T14:30:18,733 adding 'lm_eval/tasks/noreval/ask_gec/README.md' 2026-05-11T14:30:18,734 adding 'lm_eval/tasks/noreval/ask_gec/_ask_gec_yaml' 2026-05-11T14:30:18,735 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p0.yaml' 2026-05-11T14:30:18,736 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p1.yaml' 2026-05-11T14:30:18,737 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p2.yaml' 2026-05-11T14:30:18,738 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p3.yaml' 2026-05-11T14:30:18,740 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p4.yaml' 2026-05-11T14:30:18,741 adding 'lm_eval/tasks/noreval/ask_gec/errant.py' 2026-05-11T14:30:18,743 adding 'lm_eval/tasks/noreval/ncb/ncb.yaml' 2026-05-11T14:30:18,744 adding 'lm_eval/tasks/noreval/norbelebele/_norbelebele_yaml' 2026-05-11T14:30:18,745 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p0.yaml' 2026-05-11T14:30:18,747 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p1.yaml' 2026-05-11T14:30:18,748 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p2.yaml' 2026-05-11T14:30:18,749 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p3.yaml' 2026-05-11T14:30:18,750 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p4.yaml' 2026-05-11T14:30:18,752 adding 'lm_eval/tasks/noreval/norcommonsenseqa/_norcommonsenseqa_yaml' 2026-05-11T14:30:18,754 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p0.yaml' 2026-05-11T14:30:18,755 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p1.yaml' 2026-05-11T14:30:18,756 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p2.yaml' 2026-05-11T14:30:18,757 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p3.yaml' 2026-05-11T14:30:18,758 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p4.yaml' 2026-05-11T14:30:18,760 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p0.yaml' 2026-05-11T14:30:18,761 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p1.yaml' 2026-05-11T14:30:18,763 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p2.yaml' 2026-05-11T14:30:18,764 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p3.yaml' 2026-05-11T14:30:18,765 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p4.yaml' 2026-05-11T14:30:18,767 adding 'lm_eval/tasks/noreval/norec/_norec_yaml' 2026-05-11T14:30:18,768 adding 'lm_eval/tasks/noreval/norec/utils.py' 2026-05-11T14:30:18,769 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p0.yaml' 2026-05-11T14:30:18,770 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p1.yaml' 2026-05-11T14:30:18,771 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p2.yaml' 2026-05-11T14:30:18,772 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p3.yaml' 2026-05-11T14:30:18,773 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p4.yaml' 2026-05-11T14:30:18,775 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p0.yaml' 2026-05-11T14:30:18,776 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p1.yaml' 2026-05-11T14:30:18,778 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p2.yaml' 2026-05-11T14:30:18,779 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p3.yaml' 2026-05-11T14:30:18,780 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p4.yaml' 2026-05-11T14:30:18,782 adding 'lm_eval/tasks/noreval/noridiom/_noridiom_yaml' 2026-05-11T14:30:18,783 adding 'lm_eval/tasks/noreval/noridiom/utils.py' 2026-05-11T14:30:18,785 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p0.yaml' 2026-05-11T14:30:18,786 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p1.yaml' 2026-05-11T14:30:18,787 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p2.yaml' 2026-05-11T14:30:18,789 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p3.yaml' 2026-05-11T14:30:18,790 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p4.yaml' 2026-05-11T14:30:18,791 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p0.yaml' 2026-05-11T14:30:18,793 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p1.yaml' 2026-05-11T14:30:18,794 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p2.yaml' 2026-05-11T14:30:18,795 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p3.yaml' 2026-05-11T14:30:18,796 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p4.yaml' 2026-05-11T14:30:18,798 adding 'lm_eval/tasks/noreval/noropenbookqa/_noropenbookqa_yaml' 2026-05-11T14:30:18,799 adding 'lm_eval/tasks/noreval/noropenbookqa/utils.py' 2026-05-11T14:30:18,801 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p0.yaml' 2026-05-11T14:30:18,802 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p1.yaml' 2026-05-11T14:30:18,803 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p2.yaml' 2026-05-11T14:30:18,804 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p3.yaml' 2026-05-11T14:30:18,806 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p4.yaml' 2026-05-11T14:30:18,807 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p0.yaml' 2026-05-11T14:30:18,809 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p1.yaml' 2026-05-11T14:30:18,810 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p2.yaml' 2026-05-11T14:30:18,811 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p3.yaml' 2026-05-11T14:30:18,812 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p4.yaml' 2026-05-11T14:30:18,814 adding 'lm_eval/tasks/noreval/norquad/_norquad_yaml' 2026-05-11T14:30:18,815 adding 'lm_eval/tasks/noreval/norquad/norquad_p0.yaml' 2026-05-11T14:30:18,816 adding 'lm_eval/tasks/noreval/norquad/norquad_p1.yaml' 2026-05-11T14:30:18,818 adding 'lm_eval/tasks/noreval/norquad/norquad_p2.yaml' 2026-05-11T14:30:18,819 adding 'lm_eval/tasks/noreval/norquad/norquad_p3.yaml' 2026-05-11T14:30:18,820 adding 'lm_eval/tasks/noreval/norquad/norquad_p4.yaml' 2026-05-11T14:30:18,821 adding 'lm_eval/tasks/noreval/norquad/utils.py' 2026-05-11T14:30:18,822 adding 'lm_eval/tasks/noreval/norrewrite-instruct/norrewrite_instruct.yaml' 2026-05-11T14:30:18,824 adding 'lm_eval/tasks/noreval/norsumm/_norsumm_yaml' 2026-05-11T14:30:18,825 adding 'lm_eval/tasks/noreval/norsumm/utils.py' 2026-05-11T14:30:18,827 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p0.yaml' 2026-05-11T14:30:18,828 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p1.yaml' 2026-05-11T14:30:18,829 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p2.yaml' 2026-05-11T14:30:18,830 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p3.yaml' 2026-05-11T14:30:18,831 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p4.yaml' 2026-05-11T14:30:18,832 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p5.yaml' 2026-05-11T14:30:18,834 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p0.yaml' 2026-05-11T14:30:18,835 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p1.yaml' 2026-05-11T14:30:18,836 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p2.yaml' 2026-05-11T14:30:18,838 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p3.yaml' 2026-05-11T14:30:18,839 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p4.yaml' 2026-05-11T14:30:18,840 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p5.yaml' 2026-05-11T14:30:18,842 adding 'lm_eval/tasks/noreval/norsummarize-instruct/norsummarize_instruct.yaml' 2026-05-11T14:30:18,844 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/_nortruthfulqa_gen_yaml' 2026-05-11T14:30:18,845 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/utils.py' 2026-05-11T14:30:18,847 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p0.yaml' 2026-05-11T14:30:18,848 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p1.yaml' 2026-05-11T14:30:18,850 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p2.yaml' 2026-05-11T14:30:18,851 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p3.yaml' 2026-05-11T14:30:18,852 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p4.yaml' 2026-05-11T14:30:18,854 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p0.yaml' 2026-05-11T14:30:18,855 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p1.yaml' 2026-05-11T14:30:18,856 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p2.yaml' 2026-05-11T14:30:18,857 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p3.yaml' 2026-05-11T14:30:18,858 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p4.yaml' 2026-05-11T14:30:18,860 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/_nortruthfulqa_mc_yaml' 2026-05-11T14:30:18,862 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p0.yaml' 2026-05-11T14:30:18,863 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p1.yaml' 2026-05-11T14:30:18,864 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p2.yaml' 2026-05-11T14:30:18,865 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p3.yaml' 2026-05-11T14:30:18,867 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p4.yaml' 2026-05-11T14:30:18,868 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/utils.py' 2026-05-11T14:30:18,870 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p0.yaml' 2026-05-11T14:30:18,871 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p1.yaml' 2026-05-11T14:30:18,872 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p2.yaml' 2026-05-11T14:30:18,873 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p3.yaml' 2026-05-11T14:30:18,874 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p4.yaml' 2026-05-11T14:30:18,875 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/utils.py' 2026-05-11T14:30:18,877 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/_nrk_quiz_qa_yaml' 2026-05-11T14:30:18,879 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p0.yaml' 2026-05-11T14:30:18,880 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p1.yaml' 2026-05-11T14:30:18,881 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p2.yaml' 2026-05-11T14:30:18,882 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p3.yaml' 2026-05-11T14:30:18,883 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p4.yaml' 2026-05-11T14:30:18,884 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/utils.py' 2026-05-11T14:30:18,886 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p0.yaml' 2026-05-11T14:30:18,887 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p1.yaml' 2026-05-11T14:30:18,888 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p2.yaml' 2026-05-11T14:30:18,889 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p3.yaml' 2026-05-11T14:30:18,891 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p4.yaml' 2026-05-11T14:30:18,892 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/utils.py' 2026-05-11T14:30:18,894 adding 'lm_eval/tasks/noreval/tatoeba/_tatoeba_yaml' 2026-05-11T14:30:18,895 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p0.yaml' 2026-05-11T14:30:18,896 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p1.yaml' 2026-05-11T14:30:18,897 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p2.yaml' 2026-05-11T14:30:18,899 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p3.yaml' 2026-05-11T14:30:18,900 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p0.yaml' 2026-05-11T14:30:18,902 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p1.yaml' 2026-05-11T14:30:18,903 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p2.yaml' 2026-05-11T14:30:18,904 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p3.yaml' 2026-05-11T14:30:18,906 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p0.yaml' 2026-05-11T14:30:18,907 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p1.yaml' 2026-05-11T14:30:18,908 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p2.yaml' 2026-05-11T14:30:18,909 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p3.yaml' 2026-05-11T14:30:18,911 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p0.yaml' 2026-05-11T14:30:18,912 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p1.yaml' 2026-05-11T14:30:18,913 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p2.yaml' 2026-05-11T14:30:18,914 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p3.yaml' 2026-05-11T14:30:18,916 adding 'lm_eval/tasks/noticia/README.md' 2026-05-11T14:30:18,918 adding 'lm_eval/tasks/noticia/noticia.yaml' 2026-05-11T14:30:18,919 adding 'lm_eval/tasks/noticia/utils.py' 2026-05-11T14:30:18,921 adding 'lm_eval/tasks/nq_open/README.md' 2026-05-11T14:30:18,922 adding 'lm_eval/tasks/nq_open/nq_open.yaml' 2026-05-11T14:30:18,925 adding 'lm_eval/tasks/okapi/arc_multilingual/README.md' 2026-05-11T14:30:18,926 adding 'lm_eval/tasks/okapi/arc_multilingual/_arc_yaml' 2026-05-11T14:30:18,927 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml' 2026-05-11T14:30:18,928 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml' 2026-05-11T14:30:18,930 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml' 2026-05-11T14:30:18,931 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml' 2026-05-11T14:30:18,932 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml' 2026-05-11T14:30:18,933 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml' 2026-05-11T14:30:18,934 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml' 2026-05-11T14:30:18,935 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml' 2026-05-11T14:30:18,936 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml' 2026-05-11T14:30:18,937 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml' 2026-05-11T14:30:18,938 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml' 2026-05-11T14:30:18,940 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml' 2026-05-11T14:30:18,941 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml' 2026-05-11T14:30:18,942 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml' 2026-05-11T14:30:18,943 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml' 2026-05-11T14:30:18,944 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml' 2026-05-11T14:30:18,945 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml' 2026-05-11T14:30:18,946 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml' 2026-05-11T14:30:18,948 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml' 2026-05-11T14:30:18,949 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml' 2026-05-11T14:30:18,950 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml' 2026-05-11T14:30:18,951 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml' 2026-05-11T14:30:18,952 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml' 2026-05-11T14:30:18,953 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml' 2026-05-11T14:30:18,955 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml' 2026-05-11T14:30:18,956 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml' 2026-05-11T14:30:18,957 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml' 2026-05-11T14:30:18,958 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml' 2026-05-11T14:30:18,959 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml' 2026-05-11T14:30:18,960 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml' 2026-05-11T14:30:18,961 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml' 2026-05-11T14:30:18,963 adding 'lm_eval/tasks/okapi/arc_multilingual/utils.py' 2026-05-11T14:30:18,965 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/README.md' 2026-05-11T14:30:18,967 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml' 2026-05-11T14:30:18,968 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml' 2026-05-11T14:30:18,969 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml' 2026-05-11T14:30:18,970 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml' 2026-05-11T14:30:18,971 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml' 2026-05-11T14:30:18,972 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml' 2026-05-11T14:30:18,973 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml' 2026-05-11T14:30:18,975 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml' 2026-05-11T14:30:18,976 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml' 2026-05-11T14:30:18,977 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml' 2026-05-11T14:30:18,978 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml' 2026-05-11T14:30:18,979 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml' 2026-05-11T14:30:18,980 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml' 2026-05-11T14:30:18,982 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml' 2026-05-11T14:30:18,983 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml' 2026-05-11T14:30:18,984 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml' 2026-05-11T14:30:18,985 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml' 2026-05-11T14:30:18,987 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml' 2026-05-11T14:30:18,988 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml' 2026-05-11T14:30:18,989 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml' 2026-05-11T14:30:18,990 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml' 2026-05-11T14:30:18,992 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml' 2026-05-11T14:30:18,993 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml' 2026-05-11T14:30:18,994 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml' 2026-05-11T14:30:18,995 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml' 2026-05-11T14:30:18,996 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml' 2026-05-11T14:30:18,998 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml' 2026-05-11T14:30:18,999 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml' 2026-05-11T14:30:19,000 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml' 2026-05-11T14:30:19,001 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml' 2026-05-11T14:30:19,002 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml' 2026-05-11T14:30:19,003 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/utils.py' 2026-05-11T14:30:19,005 adding 'lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml' 2026-05-11T14:30:19,007 adding 'lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py' 2026-05-11T14:30:19,008 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml' 2026-05-11T14:30:19,009 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml' 2026-05-11T14:30:19,010 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml' 2026-05-11T14:30:19,011 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml' 2026-05-11T14:30:19,012 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml' 2026-05-11T14:30:19,013 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml' 2026-05-11T14:30:19,014 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml' 2026-05-11T14:30:19,016 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml' 2026-05-11T14:30:19,017 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml' 2026-05-11T14:30:19,018 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml' 2026-05-11T14:30:19,019 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml' 2026-05-11T14:30:19,020 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml' 2026-05-11T14:30:19,021 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml' 2026-05-11T14:30:19,022 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml' 2026-05-11T14:30:19,023 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml' 2026-05-11T14:30:19,025 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml' 2026-05-11T14:30:19,026 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml' 2026-05-11T14:30:19,027 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml' 2026-05-11T14:30:19,028 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml' 2026-05-11T14:30:19,030 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml' 2026-05-11T14:30:19,031 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml' 2026-05-11T14:30:19,032 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml' 2026-05-11T14:30:19,033 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml' 2026-05-11T14:30:19,034 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml' 2026-05-11T14:30:19,035 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml' 2026-05-11T14:30:19,036 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml' 2026-05-11T14:30:19,038 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml' 2026-05-11T14:30:19,039 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml' 2026-05-11T14:30:19,040 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml' 2026-05-11T14:30:19,041 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml' 2026-05-11T14:30:19,042 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml' 2026-05-11T14:30:19,043 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml' 2026-05-11T14:30:19,044 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml' 2026-05-11T14:30:19,045 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml' 2026-05-11T14:30:19,048 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/README.md' 2026-05-11T14:30:19,049 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml' 2026-05-11T14:30:19,050 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml' 2026-05-11T14:30:19,051 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml' 2026-05-11T14:30:19,052 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml' 2026-05-11T14:30:19,053 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml' 2026-05-11T14:30:19,054 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml' 2026-05-11T14:30:19,056 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml' 2026-05-11T14:30:19,057 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml' 2026-05-11T14:30:19,058 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml' 2026-05-11T14:30:19,059 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml' 2026-05-11T14:30:19,061 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml' 2026-05-11T14:30:19,062 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml' 2026-05-11T14:30:19,063 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml' 2026-05-11T14:30:19,064 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml' 2026-05-11T14:30:19,065 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml' 2026-05-11T14:30:19,067 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml' 2026-05-11T14:30:19,068 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml' 2026-05-11T14:30:19,069 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml' 2026-05-11T14:30:19,070 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml' 2026-05-11T14:30:19,071 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml' 2026-05-11T14:30:19,073 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml' 2026-05-11T14:30:19,074 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml' 2026-05-11T14:30:19,075 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml' 2026-05-11T14:30:19,076 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml' 2026-05-11T14:30:19,077 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml' 2026-05-11T14:30:19,079 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml' 2026-05-11T14:30:19,080 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml' 2026-05-11T14:30:19,081 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml' 2026-05-11T14:30:19,082 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml' 2026-05-11T14:30:19,083 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml' 2026-05-11T14:30:19,084 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml' 2026-05-11T14:30:19,086 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml' 2026-05-11T14:30:19,087 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml' 2026-05-11T14:30:19,088 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml' 2026-05-11T14:30:19,089 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml' 2026-05-11T14:30:19,090 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml' 2026-05-11T14:30:19,091 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml' 2026-05-11T14:30:19,092 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml' 2026-05-11T14:30:19,093 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml' 2026-05-11T14:30:19,095 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml' 2026-05-11T14:30:19,096 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml' 2026-05-11T14:30:19,097 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml' 2026-05-11T14:30:19,098 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml' 2026-05-11T14:30:19,099 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml' 2026-05-11T14:30:19,101 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml' 2026-05-11T14:30:19,102 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml' 2026-05-11T14:30:19,103 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml' 2026-05-11T14:30:19,104 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml' 2026-05-11T14:30:19,105 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml' 2026-05-11T14:30:19,106 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml' 2026-05-11T14:30:19,107 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml' 2026-05-11T14:30:19,109 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml' 2026-05-11T14:30:19,110 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml' 2026-05-11T14:30:19,111 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml' 2026-05-11T14:30:19,112 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml' 2026-05-11T14:30:19,113 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml' 2026-05-11T14:30:19,114 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml' 2026-05-11T14:30:19,115 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml' 2026-05-11T14:30:19,117 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml' 2026-05-11T14:30:19,118 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml' 2026-05-11T14:30:19,119 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml' 2026-05-11T14:30:19,120 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml' 2026-05-11T14:30:19,121 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml' 2026-05-11T14:30:19,123 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml' 2026-05-11T14:30:19,124 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py' 2026-05-11T14:30:19,126 adding 'lm_eval/tasks/olaph/README.md' 2026-05-11T14:30:19,127 adding 'lm_eval/tasks/olaph/olaph.yaml' 2026-05-11T14:30:19,128 adding 'lm_eval/tasks/olaph/olaph_perplexity.yaml' 2026-05-11T14:30:19,130 adding 'lm_eval/tasks/olaph/utils.py' 2026-05-11T14:30:19,131 adding 'lm_eval/tasks/olaph/utils_perplexity.py' 2026-05-11T14:30:19,133 adding 'lm_eval/tasks/openai-mmmlu/README.md' 2026-05-11T14:30:19,134 adding 'lm_eval/tasks/openai-mmmlu/_generate_configs.py' 2026-05-11T14:30:19,136 adding 'lm_eval/tasks/openai-mmmlu/languages.json' 2026-05-11T14:30:19,137 adding 'lm_eval/tasks/openai-mmmlu/subjects.json' 2026-05-11T14:30:19,187 adding 'lm_eval/tasks/openai-mmmlu/default/_default_template_yaml' 2026-05-11T14:30:19,189 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu.yaml' 2026-05-11T14:30:19,190 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy.yaml' 2026-05-11T14:30:19,191 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_humanities.yaml' 2026-05-11T14:30:19,192 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_other.yaml' 2026-05-11T14:30:19,194 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_social_sciences.yaml' 2026-05-11T14:30:19,195 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_stem.yaml' 2026-05-11T14:30:19,196 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd.yaml' 2026-05-11T14:30:19,197 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_humanities.yaml' 2026-05-11T14:30:19,198 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_other.yaml' 2026-05-11T14:30:19,199 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_social_sciences.yaml' 2026-05-11T14:30:19,200 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_stem.yaml' 2026-05-11T14:30:19,201 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de.yaml' 2026-05-11T14:30:19,203 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_humanities.yaml' 2026-05-11T14:30:19,204 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_other.yaml' 2026-05-11T14:30:19,205 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_social_sciences.yaml' 2026-05-11T14:30:19,206 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_stem.yaml' 2026-05-11T14:30:19,207 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la.yaml' 2026-05-11T14:30:19,208 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_humanities.yaml' 2026-05-11T14:30:19,210 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_other.yaml' 2026-05-11T14:30:19,211 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_social_sciences.yaml' 2026-05-11T14:30:19,212 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_stem.yaml' 2026-05-11T14:30:19,213 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr.yaml' 2026-05-11T14:30:19,214 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_humanities.yaml' 2026-05-11T14:30:19,215 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_other.yaml' 2026-05-11T14:30:19,217 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_social_sciences.yaml' 2026-05-11T14:30:19,218 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_stem.yaml' 2026-05-11T14:30:19,219 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in.yaml' 2026-05-11T14:30:19,220 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_humanities.yaml' 2026-05-11T14:30:19,221 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_other.yaml' 2026-05-11T14:30:19,222 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_social_sciences.yaml' 2026-05-11T14:30:19,223 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_stem.yaml' 2026-05-11T14:30:19,225 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id.yaml' 2026-05-11T14:30:19,226 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_humanities.yaml' 2026-05-11T14:30:19,227 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_other.yaml' 2026-05-11T14:30:19,228 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_social_sciences.yaml' 2026-05-11T14:30:19,229 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_stem.yaml' 2026-05-11T14:30:19,230 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it.yaml' 2026-05-11T14:30:19,232 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_humanities.yaml' 2026-05-11T14:30:19,233 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_other.yaml' 2026-05-11T14:30:19,234 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_social_sciences.yaml' 2026-05-11T14:30:19,235 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_stem.yaml' 2026-05-11T14:30:19,237 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp.yaml' 2026-05-11T14:30:19,238 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_humanities.yaml' 2026-05-11T14:30:19,239 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_other.yaml' 2026-05-11T14:30:19,240 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_social_sciences.yaml' 2026-05-11T14:30:19,241 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_stem.yaml' 2026-05-11T14:30:19,242 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr.yaml' 2026-05-11T14:30:19,244 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_humanities.yaml' 2026-05-11T14:30:19,245 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_other.yaml' 2026-05-11T14:30:19,246 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_social_sciences.yaml' 2026-05-11T14:30:19,247 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_stem.yaml' 2026-05-11T14:30:19,248 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br.yaml' 2026-05-11T14:30:19,249 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_humanities.yaml' 2026-05-11T14:30:19,250 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_other.yaml' 2026-05-11T14:30:19,251 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_social_sciences.yaml' 2026-05-11T14:30:19,253 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_stem.yaml' 2026-05-11T14:30:19,254 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke.yaml' 2026-05-11T14:30:19,255 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_humanities.yaml' 2026-05-11T14:30:19,256 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_other.yaml' 2026-05-11T14:30:19,257 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_social_sciences.yaml' 2026-05-11T14:30:19,258 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_stem.yaml' 2026-05-11T14:30:19,260 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng.yaml' 2026-05-11T14:30:19,261 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_humanities.yaml' 2026-05-11T14:30:19,262 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_other.yaml' 2026-05-11T14:30:19,263 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_social_sciences.yaml' 2026-05-11T14:30:19,264 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_stem.yaml' 2026-05-11T14:30:19,265 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn.yaml' 2026-05-11T14:30:19,267 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_humanities.yaml' 2026-05-11T14:30:19,268 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_other.yaml' 2026-05-11T14:30:19,269 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_social_sciences.yaml' 2026-05-11T14:30:19,270 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_stem.yaml' 2026-05-11T14:30:19,271 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_abstract_algebra.yaml' 2026-05-11T14:30:19,273 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_anatomy.yaml' 2026-05-11T14:30:19,274 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_astronomy.yaml' 2026-05-11T14:30:19,275 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_business_ethics.yaml' 2026-05-11T14:30:19,276 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_clinical_knowledge.yaml' 2026-05-11T14:30:19,277 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_biology.yaml' 2026-05-11T14:30:19,279 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_chemistry.yaml' 2026-05-11T14:30:19,280 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_computer_science.yaml' 2026-05-11T14:30:19,281 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_mathematics.yaml' 2026-05-11T14:30:19,282 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_medicine.yaml' 2026-05-11T14:30:19,283 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_physics.yaml' 2026-05-11T14:30:19,285 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_computer_security.yaml' 2026-05-11T14:30:19,286 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_conceptual_physics.yaml' 2026-05-11T14:30:19,287 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_econometrics.yaml' 2026-05-11T14:30:19,288 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_electrical_engineering.yaml' 2026-05-11T14:30:19,289 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_elementary_mathematics.yaml' 2026-05-11T14:30:19,291 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_formal_logic.yaml' 2026-05-11T14:30:19,292 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_global_facts.yaml' 2026-05-11T14:30:19,293 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_biology.yaml' 2026-05-11T14:30:19,294 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_chemistry.yaml' 2026-05-11T14:30:19,295 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_computer_science.yaml' 2026-05-11T14:30:19,296 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_european_history.yaml' 2026-05-11T14:30:19,297 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_geography.yaml' 2026-05-11T14:30:19,299 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,300 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,301 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_mathematics.yaml' 2026-05-11T14:30:19,302 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_microeconomics.yaml' 2026-05-11T14:30:19,304 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_physics.yaml' 2026-05-11T14:30:19,305 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_psychology.yaml' 2026-05-11T14:30:19,306 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_statistics.yaml' 2026-05-11T14:30:19,307 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_us_history.yaml' 2026-05-11T14:30:19,308 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_world_history.yaml' 2026-05-11T14:30:19,310 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_aging.yaml' 2026-05-11T14:30:19,311 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_sexuality.yaml' 2026-05-11T14:30:19,312 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_international_law.yaml' 2026-05-11T14:30:19,313 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_jurisprudence.yaml' 2026-05-11T14:30:19,314 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_logical_fallacies.yaml' 2026-05-11T14:30:19,316 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_machine_learning.yaml' 2026-05-11T14:30:19,317 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_management.yaml' 2026-05-11T14:30:19,318 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_marketing.yaml' 2026-05-11T14:30:19,319 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_medical_genetics.yaml' 2026-05-11T14:30:19,320 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_miscellaneous.yaml' 2026-05-11T14:30:19,321 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_disputes.yaml' 2026-05-11T14:30:19,323 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_scenarios.yaml' 2026-05-11T14:30:19,324 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_nutrition.yaml' 2026-05-11T14:30:19,325 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_philosophy.yaml' 2026-05-11T14:30:19,326 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_prehistory.yaml' 2026-05-11T14:30:19,328 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_accounting.yaml' 2026-05-11T14:30:19,329 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_law.yaml' 2026-05-11T14:30:19,330 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_medicine.yaml' 2026-05-11T14:30:19,331 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_psychology.yaml' 2026-05-11T14:30:19,332 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_public_relations.yaml' 2026-05-11T14:30:19,334 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_security_studies.yaml' 2026-05-11T14:30:19,335 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_sociology.yaml' 2026-05-11T14:30:19,336 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_us_foreign_policy.yaml' 2026-05-11T14:30:19,337 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_virology.yaml' 2026-05-11T14:30:19,338 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_world_religions.yaml' 2026-05-11T14:30:19,339 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_abstract_algebra.yaml' 2026-05-11T14:30:19,340 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_anatomy.yaml' 2026-05-11T14:30:19,342 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_astronomy.yaml' 2026-05-11T14:30:19,343 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_business_ethics.yaml' 2026-05-11T14:30:19,344 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_clinical_knowledge.yaml' 2026-05-11T14:30:19,345 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_biology.yaml' 2026-05-11T14:30:19,346 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_chemistry.yaml' 2026-05-11T14:30:19,348 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_computer_science.yaml' 2026-05-11T14:30:19,349 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_mathematics.yaml' 2026-05-11T14:30:19,350 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_medicine.yaml' 2026-05-11T14:30:19,351 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_physics.yaml' 2026-05-11T14:30:19,352 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_computer_security.yaml' 2026-05-11T14:30:19,354 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_conceptual_physics.yaml' 2026-05-11T14:30:19,355 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_econometrics.yaml' 2026-05-11T14:30:19,356 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_electrical_engineering.yaml' 2026-05-11T14:30:19,357 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_elementary_mathematics.yaml' 2026-05-11T14:30:19,358 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_formal_logic.yaml' 2026-05-11T14:30:19,359 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_global_facts.yaml' 2026-05-11T14:30:19,361 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_biology.yaml' 2026-05-11T14:30:19,362 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_chemistry.yaml' 2026-05-11T14:30:19,363 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_computer_science.yaml' 2026-05-11T14:30:19,364 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_european_history.yaml' 2026-05-11T14:30:19,365 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_geography.yaml' 2026-05-11T14:30:19,367 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,368 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,369 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_mathematics.yaml' 2026-05-11T14:30:19,370 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_microeconomics.yaml' 2026-05-11T14:30:19,371 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_physics.yaml' 2026-05-11T14:30:19,373 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_psychology.yaml' 2026-05-11T14:30:19,374 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_statistics.yaml' 2026-05-11T14:30:19,375 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_us_history.yaml' 2026-05-11T14:30:19,376 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_world_history.yaml' 2026-05-11T14:30:19,377 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_aging.yaml' 2026-05-11T14:30:19,378 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_sexuality.yaml' 2026-05-11T14:30:19,380 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_international_law.yaml' 2026-05-11T14:30:19,381 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_jurisprudence.yaml' 2026-05-11T14:30:19,382 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_logical_fallacies.yaml' 2026-05-11T14:30:19,383 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_machine_learning.yaml' 2026-05-11T14:30:19,384 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_management.yaml' 2026-05-11T14:30:19,385 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_marketing.yaml' 2026-05-11T14:30:19,386 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_medical_genetics.yaml' 2026-05-11T14:30:19,387 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_miscellaneous.yaml' 2026-05-11T14:30:19,389 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_disputes.yaml' 2026-05-11T14:30:19,390 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_scenarios.yaml' 2026-05-11T14:30:19,391 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_nutrition.yaml' 2026-05-11T14:30:19,392 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_philosophy.yaml' 2026-05-11T14:30:19,393 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_prehistory.yaml' 2026-05-11T14:30:19,395 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_accounting.yaml' 2026-05-11T14:30:19,396 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_law.yaml' 2026-05-11T14:30:19,397 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_medicine.yaml' 2026-05-11T14:30:19,398 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_psychology.yaml' 2026-05-11T14:30:19,399 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_public_relations.yaml' 2026-05-11T14:30:19,400 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_security_studies.yaml' 2026-05-11T14:30:19,402 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_sociology.yaml' 2026-05-11T14:30:19,403 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_us_foreign_policy.yaml' 2026-05-11T14:30:19,404 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_virology.yaml' 2026-05-11T14:30:19,405 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_world_religions.yaml' 2026-05-11T14:30:19,406 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_abstract_algebra.yaml' 2026-05-11T14:30:19,407 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_anatomy.yaml' 2026-05-11T14:30:19,409 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_astronomy.yaml' 2026-05-11T14:30:19,410 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_business_ethics.yaml' 2026-05-11T14:30:19,411 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_clinical_knowledge.yaml' 2026-05-11T14:30:19,412 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_biology.yaml' 2026-05-11T14:30:19,413 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_chemistry.yaml' 2026-05-11T14:30:19,414 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_computer_science.yaml' 2026-05-11T14:30:19,416 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_mathematics.yaml' 2026-05-11T14:30:19,417 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_medicine.yaml' 2026-05-11T14:30:19,418 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_physics.yaml' 2026-05-11T14:30:19,419 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_computer_security.yaml' 2026-05-11T14:30:19,420 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_conceptual_physics.yaml' 2026-05-11T14:30:19,422 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_econometrics.yaml' 2026-05-11T14:30:19,423 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_electrical_engineering.yaml' 2026-05-11T14:30:19,424 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_elementary_mathematics.yaml' 2026-05-11T14:30:19,425 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_formal_logic.yaml' 2026-05-11T14:30:19,426 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_global_facts.yaml' 2026-05-11T14:30:19,428 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_biology.yaml' 2026-05-11T14:30:19,429 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_chemistry.yaml' 2026-05-11T14:30:19,430 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_computer_science.yaml' 2026-05-11T14:30:19,431 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_european_history.yaml' 2026-05-11T14:30:19,432 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_geography.yaml' 2026-05-11T14:30:19,434 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,435 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,436 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_mathematics.yaml' 2026-05-11T14:30:19,437 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_microeconomics.yaml' 2026-05-11T14:30:19,438 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_physics.yaml' 2026-05-11T14:30:19,440 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_psychology.yaml' 2026-05-11T14:30:19,441 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_statistics.yaml' 2026-05-11T14:30:19,442 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_us_history.yaml' 2026-05-11T14:30:19,443 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_world_history.yaml' 2026-05-11T14:30:19,444 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_aging.yaml' 2026-05-11T14:30:19,445 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_sexuality.yaml' 2026-05-11T14:30:19,448 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_international_law.yaml' 2026-05-11T14:30:19,449 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_jurisprudence.yaml' 2026-05-11T14:30:19,450 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_logical_fallacies.yaml' 2026-05-11T14:30:19,451 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_machine_learning.yaml' 2026-05-11T14:30:19,454 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_management.yaml' 2026-05-11T14:30:19,455 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_marketing.yaml' 2026-05-11T14:30:19,456 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_medical_genetics.yaml' 2026-05-11T14:30:19,457 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_miscellaneous.yaml' 2026-05-11T14:30:19,458 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_disputes.yaml' 2026-05-11T14:30:19,460 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_scenarios.yaml' 2026-05-11T14:30:19,461 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_nutrition.yaml' 2026-05-11T14:30:19,462 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_philosophy.yaml' 2026-05-11T14:30:19,463 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_prehistory.yaml' 2026-05-11T14:30:19,465 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_accounting.yaml' 2026-05-11T14:30:19,466 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_law.yaml' 2026-05-11T14:30:19,467 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_medicine.yaml' 2026-05-11T14:30:19,468 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_psychology.yaml' 2026-05-11T14:30:19,469 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_public_relations.yaml' 2026-05-11T14:30:19,471 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_security_studies.yaml' 2026-05-11T14:30:19,472 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_sociology.yaml' 2026-05-11T14:30:19,473 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_us_foreign_policy.yaml' 2026-05-11T14:30:19,474 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_virology.yaml' 2026-05-11T14:30:19,476 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_world_religions.yaml' 2026-05-11T14:30:19,477 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_abstract_algebra.yaml' 2026-05-11T14:30:19,478 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_anatomy.yaml' 2026-05-11T14:30:19,479 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_astronomy.yaml' 2026-05-11T14:30:19,480 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_business_ethics.yaml' 2026-05-11T14:30:19,481 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_clinical_knowledge.yaml' 2026-05-11T14:30:19,482 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_biology.yaml' 2026-05-11T14:30:19,484 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_chemistry.yaml' 2026-05-11T14:30:19,485 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_computer_science.yaml' 2026-05-11T14:30:19,486 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_mathematics.yaml' 2026-05-11T14:30:19,487 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_medicine.yaml' 2026-05-11T14:30:19,488 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_physics.yaml' 2026-05-11T14:30:19,489 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_computer_security.yaml' 2026-05-11T14:30:19,491 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_conceptual_physics.yaml' 2026-05-11T14:30:19,492 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_econometrics.yaml' 2026-05-11T14:30:19,493 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_electrical_engineering.yaml' 2026-05-11T14:30:19,494 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_elementary_mathematics.yaml' 2026-05-11T14:30:19,495 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_formal_logic.yaml' 2026-05-11T14:30:19,496 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_global_facts.yaml' 2026-05-11T14:30:19,498 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_biology.yaml' 2026-05-11T14:30:19,499 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_chemistry.yaml' 2026-05-11T14:30:19,500 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_computer_science.yaml' 2026-05-11T14:30:19,501 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_european_history.yaml' 2026-05-11T14:30:19,502 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_geography.yaml' 2026-05-11T14:30:19,504 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,505 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,506 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_mathematics.yaml' 2026-05-11T14:30:19,507 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_microeconomics.yaml' 2026-05-11T14:30:19,508 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_physics.yaml' 2026-05-11T14:30:19,510 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_psychology.yaml' 2026-05-11T14:30:19,511 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_statistics.yaml' 2026-05-11T14:30:19,512 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_us_history.yaml' 2026-05-11T14:30:19,513 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_world_history.yaml' 2026-05-11T14:30:19,514 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_aging.yaml' 2026-05-11T14:30:19,515 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_sexuality.yaml' 2026-05-11T14:30:19,517 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_international_law.yaml' 2026-05-11T14:30:19,518 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_jurisprudence.yaml' 2026-05-11T14:30:19,519 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_logical_fallacies.yaml' 2026-05-11T14:30:19,520 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_machine_learning.yaml' 2026-05-11T14:30:19,521 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_management.yaml' 2026-05-11T14:30:19,523 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_marketing.yaml' 2026-05-11T14:30:19,524 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_medical_genetics.yaml' 2026-05-11T14:30:19,525 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_miscellaneous.yaml' 2026-05-11T14:30:19,526 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_disputes.yaml' 2026-05-11T14:30:19,527 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_scenarios.yaml' 2026-05-11T14:30:19,528 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_nutrition.yaml' 2026-05-11T14:30:19,529 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_philosophy.yaml' 2026-05-11T14:30:19,530 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_prehistory.yaml' 2026-05-11T14:30:19,531 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_accounting.yaml' 2026-05-11T14:30:19,532 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_law.yaml' 2026-05-11T14:30:19,534 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_medicine.yaml' 2026-05-11T14:30:19,535 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_psychology.yaml' 2026-05-11T14:30:19,536 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_public_relations.yaml' 2026-05-11T14:30:19,537 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_security_studies.yaml' 2026-05-11T14:30:19,538 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_sociology.yaml' 2026-05-11T14:30:19,540 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_us_foreign_policy.yaml' 2026-05-11T14:30:19,541 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_virology.yaml' 2026-05-11T14:30:19,542 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_world_religions.yaml' 2026-05-11T14:30:19,543 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_abstract_algebra.yaml' 2026-05-11T14:30:19,544 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_anatomy.yaml' 2026-05-11T14:30:19,546 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_astronomy.yaml' 2026-05-11T14:30:19,547 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_business_ethics.yaml' 2026-05-11T14:30:19,548 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_clinical_knowledge.yaml' 2026-05-11T14:30:19,549 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_biology.yaml' 2026-05-11T14:30:19,550 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_chemistry.yaml' 2026-05-11T14:30:19,552 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_computer_science.yaml' 2026-05-11T14:30:19,553 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_mathematics.yaml' 2026-05-11T14:30:19,554 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_medicine.yaml' 2026-05-11T14:30:19,555 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_physics.yaml' 2026-05-11T14:30:19,556 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_computer_security.yaml' 2026-05-11T14:30:19,558 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_conceptual_physics.yaml' 2026-05-11T14:30:19,559 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_econometrics.yaml' 2026-05-11T14:30:19,560 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_electrical_engineering.yaml' 2026-05-11T14:30:19,561 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_elementary_mathematics.yaml' 2026-05-11T14:30:19,562 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_formal_logic.yaml' 2026-05-11T14:30:19,564 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_global_facts.yaml' 2026-05-11T14:30:19,565 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_biology.yaml' 2026-05-11T14:30:19,566 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_chemistry.yaml' 2026-05-11T14:30:19,567 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_computer_science.yaml' 2026-05-11T14:30:19,568 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_european_history.yaml' 2026-05-11T14:30:19,570 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_geography.yaml' 2026-05-11T14:30:19,571 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,572 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,573 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_mathematics.yaml' 2026-05-11T14:30:19,574 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_microeconomics.yaml' 2026-05-11T14:30:19,575 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_physics.yaml' 2026-05-11T14:30:19,576 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_psychology.yaml' 2026-05-11T14:30:19,577 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_statistics.yaml' 2026-05-11T14:30:19,579 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_us_history.yaml' 2026-05-11T14:30:19,580 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_world_history.yaml' 2026-05-11T14:30:19,581 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_aging.yaml' 2026-05-11T14:30:19,582 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_sexuality.yaml' 2026-05-11T14:30:19,583 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_international_law.yaml' 2026-05-11T14:30:19,584 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_jurisprudence.yaml' 2026-05-11T14:30:19,586 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_logical_fallacies.yaml' 2026-05-11T14:30:19,587 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_machine_learning.yaml' 2026-05-11T14:30:19,588 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_management.yaml' 2026-05-11T14:30:19,589 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_marketing.yaml' 2026-05-11T14:30:19,590 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_medical_genetics.yaml' 2026-05-11T14:30:19,592 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_miscellaneous.yaml' 2026-05-11T14:30:19,593 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_disputes.yaml' 2026-05-11T14:30:19,594 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_scenarios.yaml' 2026-05-11T14:30:19,595 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_nutrition.yaml' 2026-05-11T14:30:19,596 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_philosophy.yaml' 2026-05-11T14:30:19,598 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_prehistory.yaml' 2026-05-11T14:30:19,599 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_accounting.yaml' 2026-05-11T14:30:19,600 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_law.yaml' 2026-05-11T14:30:19,601 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_medicine.yaml' 2026-05-11T14:30:19,602 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_psychology.yaml' 2026-05-11T14:30:19,604 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_public_relations.yaml' 2026-05-11T14:30:19,605 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_security_studies.yaml' 2026-05-11T14:30:19,606 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_sociology.yaml' 2026-05-11T14:30:19,607 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_us_foreign_policy.yaml' 2026-05-11T14:30:19,609 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_virology.yaml' 2026-05-11T14:30:19,610 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_world_religions.yaml' 2026-05-11T14:30:19,611 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_abstract_algebra.yaml' 2026-05-11T14:30:19,612 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_anatomy.yaml' 2026-05-11T14:30:19,613 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_astronomy.yaml' 2026-05-11T14:30:19,615 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_business_ethics.yaml' 2026-05-11T14:30:19,616 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_clinical_knowledge.yaml' 2026-05-11T14:30:19,617 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_biology.yaml' 2026-05-11T14:30:19,618 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_chemistry.yaml' 2026-05-11T14:30:19,619 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_computer_science.yaml' 2026-05-11T14:30:19,620 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_mathematics.yaml' 2026-05-11T14:30:19,621 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_medicine.yaml' 2026-05-11T14:30:19,622 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_physics.yaml' 2026-05-11T14:30:19,624 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_computer_security.yaml' 2026-05-11T14:30:19,625 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_conceptual_physics.yaml' 2026-05-11T14:30:19,626 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_econometrics.yaml' 2026-05-11T14:30:19,627 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_electrical_engineering.yaml' 2026-05-11T14:30:19,628 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_elementary_mathematics.yaml' 2026-05-11T14:30:19,629 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_formal_logic.yaml' 2026-05-11T14:30:19,631 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_global_facts.yaml' 2026-05-11T14:30:19,632 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_biology.yaml' 2026-05-11T14:30:19,633 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_chemistry.yaml' 2026-05-11T14:30:19,634 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_computer_science.yaml' 2026-05-11T14:30:19,635 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_european_history.yaml' 2026-05-11T14:30:19,637 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_geography.yaml' 2026-05-11T14:30:19,638 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,639 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,640 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_mathematics.yaml' 2026-05-11T14:30:19,641 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_microeconomics.yaml' 2026-05-11T14:30:19,643 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_physics.yaml' 2026-05-11T14:30:19,644 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_psychology.yaml' 2026-05-11T14:30:19,645 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_statistics.yaml' 2026-05-11T14:30:19,646 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_us_history.yaml' 2026-05-11T14:30:19,647 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_world_history.yaml' 2026-05-11T14:30:19,649 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_aging.yaml' 2026-05-11T14:30:19,650 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_sexuality.yaml' 2026-05-11T14:30:19,651 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_international_law.yaml' 2026-05-11T14:30:19,652 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_jurisprudence.yaml' 2026-05-11T14:30:19,653 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_logical_fallacies.yaml' 2026-05-11T14:30:19,655 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_machine_learning.yaml' 2026-05-11T14:30:19,656 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_management.yaml' 2026-05-11T14:30:19,657 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_marketing.yaml' 2026-05-11T14:30:19,658 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_medical_genetics.yaml' 2026-05-11T14:30:19,659 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_miscellaneous.yaml' 2026-05-11T14:30:19,661 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_disputes.yaml' 2026-05-11T14:30:19,662 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_scenarios.yaml' 2026-05-11T14:30:19,663 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_nutrition.yaml' 2026-05-11T14:30:19,664 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_philosophy.yaml' 2026-05-11T14:30:19,665 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_prehistory.yaml' 2026-05-11T14:30:19,666 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_accounting.yaml' 2026-05-11T14:30:19,667 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_law.yaml' 2026-05-11T14:30:19,668 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_medicine.yaml' 2026-05-11T14:30:19,670 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_psychology.yaml' 2026-05-11T14:30:19,671 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_public_relations.yaml' 2026-05-11T14:30:19,672 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_security_studies.yaml' 2026-05-11T14:30:19,673 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_sociology.yaml' 2026-05-11T14:30:19,674 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_us_foreign_policy.yaml' 2026-05-11T14:30:19,676 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_virology.yaml' 2026-05-11T14:30:19,677 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_world_religions.yaml' 2026-05-11T14:30:19,678 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_abstract_algebra.yaml' 2026-05-11T14:30:19,679 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_anatomy.yaml' 2026-05-11T14:30:19,680 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_astronomy.yaml' 2026-05-11T14:30:19,681 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_business_ethics.yaml' 2026-05-11T14:30:19,682 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_clinical_knowledge.yaml' 2026-05-11T14:30:19,684 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_biology.yaml' 2026-05-11T14:30:19,685 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_chemistry.yaml' 2026-05-11T14:30:19,686 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_computer_science.yaml' 2026-05-11T14:30:19,687 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_mathematics.yaml' 2026-05-11T14:30:19,688 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_medicine.yaml' 2026-05-11T14:30:19,690 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_physics.yaml' 2026-05-11T14:30:19,691 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_computer_security.yaml' 2026-05-11T14:30:19,692 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_conceptual_physics.yaml' 2026-05-11T14:30:19,693 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_econometrics.yaml' 2026-05-11T14:30:19,694 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_electrical_engineering.yaml' 2026-05-11T14:30:19,696 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_elementary_mathematics.yaml' 2026-05-11T14:30:19,697 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_formal_logic.yaml' 2026-05-11T14:30:19,698 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_global_facts.yaml' 2026-05-11T14:30:19,699 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_biology.yaml' 2026-05-11T14:30:19,700 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_chemistry.yaml' 2026-05-11T14:30:19,702 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_computer_science.yaml' 2026-05-11T14:30:19,703 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_european_history.yaml' 2026-05-11T14:30:19,704 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_geography.yaml' 2026-05-11T14:30:19,705 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,707 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,708 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_mathematics.yaml' 2026-05-11T14:30:19,709 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_microeconomics.yaml' 2026-05-11T14:30:19,710 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_physics.yaml' 2026-05-11T14:30:19,711 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_psychology.yaml' 2026-05-11T14:30:19,712 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_statistics.yaml' 2026-05-11T14:30:19,713 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_us_history.yaml' 2026-05-11T14:30:19,715 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_world_history.yaml' 2026-05-11T14:30:19,716 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_aging.yaml' 2026-05-11T14:30:19,717 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_sexuality.yaml' 2026-05-11T14:30:19,718 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_international_law.yaml' 2026-05-11T14:30:19,719 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_jurisprudence.yaml' 2026-05-11T14:30:19,720 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_logical_fallacies.yaml' 2026-05-11T14:30:19,722 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_machine_learning.yaml' 2026-05-11T14:30:19,723 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_management.yaml' 2026-05-11T14:30:19,724 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_marketing.yaml' 2026-05-11T14:30:19,725 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_medical_genetics.yaml' 2026-05-11T14:30:19,726 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_miscellaneous.yaml' 2026-05-11T14:30:19,727 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_disputes.yaml' 2026-05-11T14:30:19,729 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_scenarios.yaml' 2026-05-11T14:30:19,730 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_nutrition.yaml' 2026-05-11T14:30:19,731 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_philosophy.yaml' 2026-05-11T14:30:19,732 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_prehistory.yaml' 2026-05-11T14:30:19,733 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_accounting.yaml' 2026-05-11T14:30:19,735 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_law.yaml' 2026-05-11T14:30:19,736 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_medicine.yaml' 2026-05-11T14:30:19,737 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_psychology.yaml' 2026-05-11T14:30:19,738 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_public_relations.yaml' 2026-05-11T14:30:19,740 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_security_studies.yaml' 2026-05-11T14:30:19,741 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_sociology.yaml' 2026-05-11T14:30:19,742 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_us_foreign_policy.yaml' 2026-05-11T14:30:19,743 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_virology.yaml' 2026-05-11T14:30:19,744 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_world_religions.yaml' 2026-05-11T14:30:19,746 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_abstract_algebra.yaml' 2026-05-11T14:30:19,747 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_anatomy.yaml' 2026-05-11T14:30:19,748 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_astronomy.yaml' 2026-05-11T14:30:19,749 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_business_ethics.yaml' 2026-05-11T14:30:19,750 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_clinical_knowledge.yaml' 2026-05-11T14:30:19,752 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_biology.yaml' 2026-05-11T14:30:19,753 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_chemistry.yaml' 2026-05-11T14:30:19,754 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_computer_science.yaml' 2026-05-11T14:30:19,755 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_mathematics.yaml' 2026-05-11T14:30:19,756 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_medicine.yaml' 2026-05-11T14:30:19,757 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_physics.yaml' 2026-05-11T14:30:19,758 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_computer_security.yaml' 2026-05-11T14:30:19,759 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_conceptual_physics.yaml' 2026-05-11T14:30:19,760 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_econometrics.yaml' 2026-05-11T14:30:19,762 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_electrical_engineering.yaml' 2026-05-11T14:30:19,763 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_elementary_mathematics.yaml' 2026-05-11T14:30:19,764 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_formal_logic.yaml' 2026-05-11T14:30:19,765 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_global_facts.yaml' 2026-05-11T14:30:19,766 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_biology.yaml' 2026-05-11T14:30:19,767 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_chemistry.yaml' 2026-05-11T14:30:19,769 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_computer_science.yaml' 2026-05-11T14:30:19,770 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_european_history.yaml' 2026-05-11T14:30:19,771 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_geography.yaml' 2026-05-11T14:30:19,772 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,773 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,775 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_mathematics.yaml' 2026-05-11T14:30:19,776 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_microeconomics.yaml' 2026-05-11T14:30:19,777 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_physics.yaml' 2026-05-11T14:30:19,778 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_psychology.yaml' 2026-05-11T14:30:19,779 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_statistics.yaml' 2026-05-11T14:30:19,780 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_us_history.yaml' 2026-05-11T14:30:19,782 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_world_history.yaml' 2026-05-11T14:30:19,783 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_aging.yaml' 2026-05-11T14:30:19,784 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_sexuality.yaml' 2026-05-11T14:30:19,785 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_international_law.yaml' 2026-05-11T14:30:19,786 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_jurisprudence.yaml' 2026-05-11T14:30:19,787 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_logical_fallacies.yaml' 2026-05-11T14:30:19,789 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_machine_learning.yaml' 2026-05-11T14:30:19,790 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_management.yaml' 2026-05-11T14:30:19,791 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_marketing.yaml' 2026-05-11T14:30:19,792 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_medical_genetics.yaml' 2026-05-11T14:30:19,793 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_miscellaneous.yaml' 2026-05-11T14:30:19,795 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_disputes.yaml' 2026-05-11T14:30:19,796 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_scenarios.yaml' 2026-05-11T14:30:19,797 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_nutrition.yaml' 2026-05-11T14:30:19,798 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_philosophy.yaml' 2026-05-11T14:30:19,799 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_prehistory.yaml' 2026-05-11T14:30:19,801 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_accounting.yaml' 2026-05-11T14:30:19,802 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_law.yaml' 2026-05-11T14:30:19,803 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_medicine.yaml' 2026-05-11T14:30:19,804 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_psychology.yaml' 2026-05-11T14:30:19,805 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_public_relations.yaml' 2026-05-11T14:30:19,806 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_security_studies.yaml' 2026-05-11T14:30:19,808 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_sociology.yaml' 2026-05-11T14:30:19,809 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_us_foreign_policy.yaml' 2026-05-11T14:30:19,810 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_virology.yaml' 2026-05-11T14:30:19,811 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_world_religions.yaml' 2026-05-11T14:30:19,812 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_abstract_algebra.yaml' 2026-05-11T14:30:19,813 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_anatomy.yaml' 2026-05-11T14:30:19,815 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_astronomy.yaml' 2026-05-11T14:30:19,816 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_business_ethics.yaml' 2026-05-11T14:30:19,817 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_clinical_knowledge.yaml' 2026-05-11T14:30:19,818 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_biology.yaml' 2026-05-11T14:30:19,819 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_chemistry.yaml' 2026-05-11T14:30:19,820 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_computer_science.yaml' 2026-05-11T14:30:19,822 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_mathematics.yaml' 2026-05-11T14:30:19,823 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_medicine.yaml' 2026-05-11T14:30:19,824 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_physics.yaml' 2026-05-11T14:30:19,825 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_computer_security.yaml' 2026-05-11T14:30:19,826 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_conceptual_physics.yaml' 2026-05-11T14:30:19,828 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_econometrics.yaml' 2026-05-11T14:30:19,829 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_electrical_engineering.yaml' 2026-05-11T14:30:19,830 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_elementary_mathematics.yaml' 2026-05-11T14:30:19,831 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_formal_logic.yaml' 2026-05-11T14:30:19,833 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_global_facts.yaml' 2026-05-11T14:30:19,834 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_biology.yaml' 2026-05-11T14:30:19,836 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_chemistry.yaml' 2026-05-11T14:30:19,837 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_computer_science.yaml' 2026-05-11T14:30:19,838 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_european_history.yaml' 2026-05-11T14:30:19,839 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_geography.yaml' 2026-05-11T14:30:19,841 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,842 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,843 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_mathematics.yaml' 2026-05-11T14:30:19,845 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_microeconomics.yaml' 2026-05-11T14:30:19,846 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_physics.yaml' 2026-05-11T14:30:19,847 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_psychology.yaml' 2026-05-11T14:30:19,848 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_statistics.yaml' 2026-05-11T14:30:19,849 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_us_history.yaml' 2026-05-11T14:30:19,850 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_world_history.yaml' 2026-05-11T14:30:19,851 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_aging.yaml' 2026-05-11T14:30:19,853 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_sexuality.yaml' 2026-05-11T14:30:19,854 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_international_law.yaml' 2026-05-11T14:30:19,855 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_jurisprudence.yaml' 2026-05-11T14:30:19,856 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_logical_fallacies.yaml' 2026-05-11T14:30:19,857 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_machine_learning.yaml' 2026-05-11T14:30:19,859 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_management.yaml' 2026-05-11T14:30:19,860 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_marketing.yaml' 2026-05-11T14:30:19,861 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_medical_genetics.yaml' 2026-05-11T14:30:19,862 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_miscellaneous.yaml' 2026-05-11T14:30:19,863 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_disputes.yaml' 2026-05-11T14:30:19,864 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_scenarios.yaml' 2026-05-11T14:30:19,866 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_nutrition.yaml' 2026-05-11T14:30:19,867 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_philosophy.yaml' 2026-05-11T14:30:19,868 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_prehistory.yaml' 2026-05-11T14:30:19,869 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_accounting.yaml' 2026-05-11T14:30:19,870 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_law.yaml' 2026-05-11T14:30:19,872 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_medicine.yaml' 2026-05-11T14:30:19,873 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_psychology.yaml' 2026-05-11T14:30:19,874 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_public_relations.yaml' 2026-05-11T14:30:19,875 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_security_studies.yaml' 2026-05-11T14:30:19,876 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_sociology.yaml' 2026-05-11T14:30:19,877 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_us_foreign_policy.yaml' 2026-05-11T14:30:19,879 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_virology.yaml' 2026-05-11T14:30:19,880 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_world_religions.yaml' 2026-05-11T14:30:19,881 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_abstract_algebra.yaml' 2026-05-11T14:30:19,882 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_anatomy.yaml' 2026-05-11T14:30:19,883 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_astronomy.yaml' 2026-05-11T14:30:19,885 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_business_ethics.yaml' 2026-05-11T14:30:19,886 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_clinical_knowledge.yaml' 2026-05-11T14:30:19,887 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_biology.yaml' 2026-05-11T14:30:19,888 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_chemistry.yaml' 2026-05-11T14:30:19,889 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_computer_science.yaml' 2026-05-11T14:30:19,890 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_mathematics.yaml' 2026-05-11T14:30:19,892 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_medicine.yaml' 2026-05-11T14:30:19,893 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_physics.yaml' 2026-05-11T14:30:19,894 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_computer_security.yaml' 2026-05-11T14:30:19,895 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_conceptual_physics.yaml' 2026-05-11T14:30:19,896 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_econometrics.yaml' 2026-05-11T14:30:19,897 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_electrical_engineering.yaml' 2026-05-11T14:30:19,898 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_elementary_mathematics.yaml' 2026-05-11T14:30:19,899 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_formal_logic.yaml' 2026-05-11T14:30:19,900 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_global_facts.yaml' 2026-05-11T14:30:19,902 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_biology.yaml' 2026-05-11T14:30:19,903 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_chemistry.yaml' 2026-05-11T14:30:19,904 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_computer_science.yaml' 2026-05-11T14:30:19,905 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_european_history.yaml' 2026-05-11T14:30:19,906 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_geography.yaml' 2026-05-11T14:30:19,907 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,908 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,909 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_mathematics.yaml' 2026-05-11T14:30:19,911 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_microeconomics.yaml' 2026-05-11T14:30:19,912 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_physics.yaml' 2026-05-11T14:30:19,913 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_psychology.yaml' 2026-05-11T14:30:19,914 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_statistics.yaml' 2026-05-11T14:30:19,915 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_us_history.yaml' 2026-05-11T14:30:19,917 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_world_history.yaml' 2026-05-11T14:30:19,918 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_aging.yaml' 2026-05-11T14:30:19,919 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_sexuality.yaml' 2026-05-11T14:30:19,920 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_international_law.yaml' 2026-05-11T14:30:19,921 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_jurisprudence.yaml' 2026-05-11T14:30:19,923 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_logical_fallacies.yaml' 2026-05-11T14:30:19,924 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_machine_learning.yaml' 2026-05-11T14:30:19,925 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_management.yaml' 2026-05-11T14:30:19,926 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_marketing.yaml' 2026-05-11T14:30:19,927 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_medical_genetics.yaml' 2026-05-11T14:30:19,929 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_miscellaneous.yaml' 2026-05-11T14:30:19,930 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_disputes.yaml' 2026-05-11T14:30:19,931 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_scenarios.yaml' 2026-05-11T14:30:19,932 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_nutrition.yaml' 2026-05-11T14:30:19,933 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_philosophy.yaml' 2026-05-11T14:30:19,935 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_prehistory.yaml' 2026-05-11T14:30:19,936 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_accounting.yaml' 2026-05-11T14:30:19,937 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_law.yaml' 2026-05-11T14:30:19,938 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_medicine.yaml' 2026-05-11T14:30:19,939 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_psychology.yaml' 2026-05-11T14:30:19,940 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_public_relations.yaml' 2026-05-11T14:30:19,941 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_security_studies.yaml' 2026-05-11T14:30:19,943 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_sociology.yaml' 2026-05-11T14:30:19,944 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_us_foreign_policy.yaml' 2026-05-11T14:30:19,945 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_virology.yaml' 2026-05-11T14:30:19,946 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_world_religions.yaml' 2026-05-11T14:30:19,947 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_abstract_algebra.yaml' 2026-05-11T14:30:19,949 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_anatomy.yaml' 2026-05-11T14:30:19,950 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_astronomy.yaml' 2026-05-11T14:30:19,951 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_business_ethics.yaml' 2026-05-11T14:30:19,952 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_clinical_knowledge.yaml' 2026-05-11T14:30:19,953 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_biology.yaml' 2026-05-11T14:30:19,955 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_chemistry.yaml' 2026-05-11T14:30:19,956 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_computer_science.yaml' 2026-05-11T14:30:19,957 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_mathematics.yaml' 2026-05-11T14:30:19,958 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_medicine.yaml' 2026-05-11T14:30:19,959 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_physics.yaml' 2026-05-11T14:30:19,961 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_computer_security.yaml' 2026-05-11T14:30:19,962 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_conceptual_physics.yaml' 2026-05-11T14:30:19,963 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_econometrics.yaml' 2026-05-11T14:30:19,964 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_electrical_engineering.yaml' 2026-05-11T14:30:19,965 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_elementary_mathematics.yaml' 2026-05-11T14:30:19,967 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_formal_logic.yaml' 2026-05-11T14:30:19,968 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_global_facts.yaml' 2026-05-11T14:30:19,969 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_biology.yaml' 2026-05-11T14:30:19,970 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_chemistry.yaml' 2026-05-11T14:30:19,972 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_computer_science.yaml' 2026-05-11T14:30:19,973 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_european_history.yaml' 2026-05-11T14:30:19,974 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_geography.yaml' 2026-05-11T14:30:19,975 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_government_and_politics.yaml' 2026-05-11T14:30:19,976 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_macroeconomics.yaml' 2026-05-11T14:30:19,978 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_mathematics.yaml' 2026-05-11T14:30:19,979 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_microeconomics.yaml' 2026-05-11T14:30:19,980 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_physics.yaml' 2026-05-11T14:30:19,981 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_psychology.yaml' 2026-05-11T14:30:19,983 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_statistics.yaml' 2026-05-11T14:30:19,984 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_us_history.yaml' 2026-05-11T14:30:19,985 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_world_history.yaml' 2026-05-11T14:30:19,986 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_aging.yaml' 2026-05-11T14:30:19,987 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_sexuality.yaml' 2026-05-11T14:30:19,988 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_international_law.yaml' 2026-05-11T14:30:19,989 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_jurisprudence.yaml' 2026-05-11T14:30:19,991 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_logical_fallacies.yaml' 2026-05-11T14:30:19,992 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_machine_learning.yaml' 2026-05-11T14:30:19,993 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_management.yaml' 2026-05-11T14:30:19,994 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_marketing.yaml' 2026-05-11T14:30:19,995 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_medical_genetics.yaml' 2026-05-11T14:30:19,997 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_miscellaneous.yaml' 2026-05-11T14:30:19,998 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_disputes.yaml' 2026-05-11T14:30:19,999 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_scenarios.yaml' 2026-05-11T14:30:20,000 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_nutrition.yaml' 2026-05-11T14:30:20,001 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_philosophy.yaml' 2026-05-11T14:30:20,003 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_prehistory.yaml' 2026-05-11T14:30:20,004 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_accounting.yaml' 2026-05-11T14:30:20,005 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_law.yaml' 2026-05-11T14:30:20,006 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_medicine.yaml' 2026-05-11T14:30:20,007 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_psychology.yaml' 2026-05-11T14:30:20,009 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_public_relations.yaml' 2026-05-11T14:30:20,010 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_security_studies.yaml' 2026-05-11T14:30:20,011 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_sociology.yaml' 2026-05-11T14:30:20,012 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_us_foreign_policy.yaml' 2026-05-11T14:30:20,013 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_virology.yaml' 2026-05-11T14:30:20,015 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_world_religions.yaml' 2026-05-11T14:30:20,016 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_abstract_algebra.yaml' 2026-05-11T14:30:20,017 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_anatomy.yaml' 2026-05-11T14:30:20,018 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_astronomy.yaml' 2026-05-11T14:30:20,020 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_business_ethics.yaml' 2026-05-11T14:30:20,021 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_clinical_knowledge.yaml' 2026-05-11T14:30:20,022 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_biology.yaml' 2026-05-11T14:30:20,023 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_chemistry.yaml' 2026-05-11T14:30:20,024 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_computer_science.yaml' 2026-05-11T14:30:20,025 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_mathematics.yaml' 2026-05-11T14:30:20,027 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_medicine.yaml' 2026-05-11T14:30:20,028 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_physics.yaml' 2026-05-11T14:30:20,029 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_computer_security.yaml' 2026-05-11T14:30:20,031 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_conceptual_physics.yaml' 2026-05-11T14:30:20,032 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_econometrics.yaml' 2026-05-11T14:30:20,033 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_electrical_engineering.yaml' 2026-05-11T14:30:20,034 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_elementary_mathematics.yaml' 2026-05-11T14:30:20,035 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_formal_logic.yaml' 2026-05-11T14:30:20,037 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_global_facts.yaml' 2026-05-11T14:30:20,038 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_biology.yaml' 2026-05-11T14:30:20,039 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_chemistry.yaml' 2026-05-11T14:30:20,040 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_computer_science.yaml' 2026-05-11T14:30:20,041 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_european_history.yaml' 2026-05-11T14:30:20,043 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_geography.yaml' 2026-05-11T14:30:20,044 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_government_and_politics.yaml' 2026-05-11T14:30:20,045 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_macroeconomics.yaml' 2026-05-11T14:30:20,046 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_mathematics.yaml' 2026-05-11T14:30:20,047 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_microeconomics.yaml' 2026-05-11T14:30:20,049 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_physics.yaml' 2026-05-11T14:30:20,050 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_psychology.yaml' 2026-05-11T14:30:20,051 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_statistics.yaml' 2026-05-11T14:30:20,052 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_us_history.yaml' 2026-05-11T14:30:20,053 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_world_history.yaml' 2026-05-11T14:30:20,055 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_aging.yaml' 2026-05-11T14:30:20,056 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_sexuality.yaml' 2026-05-11T14:30:20,057 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_international_law.yaml' 2026-05-11T14:30:20,058 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_jurisprudence.yaml' 2026-05-11T14:30:20,059 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_logical_fallacies.yaml' 2026-05-11T14:30:20,061 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_machine_learning.yaml' 2026-05-11T14:30:20,062 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_management.yaml' 2026-05-11T14:30:20,063 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_marketing.yaml' 2026-05-11T14:30:20,064 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_medical_genetics.yaml' 2026-05-11T14:30:20,065 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_miscellaneous.yaml' 2026-05-11T14:30:20,067 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_disputes.yaml' 2026-05-11T14:30:20,068 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_scenarios.yaml' 2026-05-11T14:30:20,069 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_nutrition.yaml' 2026-05-11T14:30:20,070 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_philosophy.yaml' 2026-05-11T14:30:20,071 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_prehistory.yaml' 2026-05-11T14:30:20,073 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_accounting.yaml' 2026-05-11T14:30:20,074 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_law.yaml' 2026-05-11T14:30:20,075 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_medicine.yaml' 2026-05-11T14:30:20,076 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_psychology.yaml' 2026-05-11T14:30:20,078 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_public_relations.yaml' 2026-05-11T14:30:20,079 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_security_studies.yaml' 2026-05-11T14:30:20,080 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_sociology.yaml' 2026-05-11T14:30:20,081 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_us_foreign_policy.yaml' 2026-05-11T14:30:20,082 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_virology.yaml' 2026-05-11T14:30:20,083 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_world_religions.yaml' 2026-05-11T14:30:20,084 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_abstract_algebra.yaml' 2026-05-11T14:30:20,085 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_anatomy.yaml' 2026-05-11T14:30:20,087 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_astronomy.yaml' 2026-05-11T14:30:20,088 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_business_ethics.yaml' 2026-05-11T14:30:20,089 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_clinical_knowledge.yaml' 2026-05-11T14:30:20,090 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_biology.yaml' 2026-05-11T14:30:20,092 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_chemistry.yaml' 2026-05-11T14:30:20,093 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_computer_science.yaml' 2026-05-11T14:30:20,094 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_mathematics.yaml' 2026-05-11T14:30:20,095 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_medicine.yaml' 2026-05-11T14:30:20,096 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_physics.yaml' 2026-05-11T14:30:20,098 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_computer_security.yaml' 2026-05-11T14:30:20,099 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_conceptual_physics.yaml' 2026-05-11T14:30:20,100 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_econometrics.yaml' 2026-05-11T14:30:20,101 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_electrical_engineering.yaml' 2026-05-11T14:30:20,102 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_elementary_mathematics.yaml' 2026-05-11T14:30:20,103 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_formal_logic.yaml' 2026-05-11T14:30:20,105 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_global_facts.yaml' 2026-05-11T14:30:20,106 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_biology.yaml' 2026-05-11T14:30:20,107 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_chemistry.yaml' 2026-05-11T14:30:20,108 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_computer_science.yaml' 2026-05-11T14:30:20,110 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_european_history.yaml' 2026-05-11T14:30:20,111 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_geography.yaml' 2026-05-11T14:30:20,112 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_government_and_politics.yaml' 2026-05-11T14:30:20,113 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_macroeconomics.yaml' 2026-05-11T14:30:20,114 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_mathematics.yaml' 2026-05-11T14:30:20,115 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_microeconomics.yaml' 2026-05-11T14:30:20,117 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_physics.yaml' 2026-05-11T14:30:20,118 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_psychology.yaml' 2026-05-11T14:30:20,119 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_statistics.yaml' 2026-05-11T14:30:20,120 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_us_history.yaml' 2026-05-11T14:30:20,122 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_world_history.yaml' 2026-05-11T14:30:20,123 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_aging.yaml' 2026-05-11T14:30:20,124 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_sexuality.yaml' 2026-05-11T14:30:20,125 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_international_law.yaml' 2026-05-11T14:30:20,126 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_jurisprudence.yaml' 2026-05-11T14:30:20,127 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_logical_fallacies.yaml' 2026-05-11T14:30:20,128 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_machine_learning.yaml' 2026-05-11T14:30:20,130 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_management.yaml' 2026-05-11T14:30:20,131 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_marketing.yaml' 2026-05-11T14:30:20,132 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_medical_genetics.yaml' 2026-05-11T14:30:20,133 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_miscellaneous.yaml' 2026-05-11T14:30:20,134 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_disputes.yaml' 2026-05-11T14:30:20,136 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_scenarios.yaml' 2026-05-11T14:30:20,137 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_nutrition.yaml' 2026-05-11T14:30:20,138 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_philosophy.yaml' 2026-05-11T14:30:20,139 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_prehistory.yaml' 2026-05-11T14:30:20,140 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_accounting.yaml' 2026-05-11T14:30:20,142 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_law.yaml' 2026-05-11T14:30:20,143 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_medicine.yaml' 2026-05-11T14:30:20,144 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_psychology.yaml' 2026-05-11T14:30:20,145 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_public_relations.yaml' 2026-05-11T14:30:20,147 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_security_studies.yaml' 2026-05-11T14:30:20,148 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_sociology.yaml' 2026-05-11T14:30:20,149 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_us_foreign_policy.yaml' 2026-05-11T14:30:20,150 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_virology.yaml' 2026-05-11T14:30:20,152 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_world_religions.yaml' 2026-05-11T14:30:20,153 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_abstract_algebra.yaml' 2026-05-11T14:30:20,154 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_anatomy.yaml' 2026-05-11T14:30:20,155 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_astronomy.yaml' 2026-05-11T14:30:20,156 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_business_ethics.yaml' 2026-05-11T14:30:20,158 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_clinical_knowledge.yaml' 2026-05-11T14:30:20,159 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_biology.yaml' 2026-05-11T14:30:20,160 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_chemistry.yaml' 2026-05-11T14:30:20,161 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_computer_science.yaml' 2026-05-11T14:30:20,162 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_mathematics.yaml' 2026-05-11T14:30:20,164 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_medicine.yaml' 2026-05-11T14:30:20,165 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_physics.yaml' 2026-05-11T14:30:20,166 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_computer_security.yaml' 2026-05-11T14:30:20,167 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_conceptual_physics.yaml' 2026-05-11T14:30:20,168 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_econometrics.yaml' 2026-05-11T14:30:20,170 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_electrical_engineering.yaml' 2026-05-11T14:30:20,171 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_elementary_mathematics.yaml' 2026-05-11T14:30:20,172 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_formal_logic.yaml' 2026-05-11T14:30:20,173 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_global_facts.yaml' 2026-05-11T14:30:20,174 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_biology.yaml' 2026-05-11T14:30:20,175 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_chemistry.yaml' 2026-05-11T14:30:20,176 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_computer_science.yaml' 2026-05-11T14:30:20,178 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_european_history.yaml' 2026-05-11T14:30:20,179 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_geography.yaml' 2026-05-11T14:30:20,180 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_government_and_politics.yaml' 2026-05-11T14:30:20,181 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_macroeconomics.yaml' 2026-05-11T14:30:20,184 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_mathematics.yaml' 2026-05-11T14:30:20,185 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_microeconomics.yaml' 2026-05-11T14:30:20,187 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_physics.yaml' 2026-05-11T14:30:20,188 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_psychology.yaml' 2026-05-11T14:30:20,189 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_statistics.yaml' 2026-05-11T14:30:20,190 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_us_history.yaml' 2026-05-11T14:30:20,191 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_world_history.yaml' 2026-05-11T14:30:20,193 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_aging.yaml' 2026-05-11T14:30:20,194 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_sexuality.yaml' 2026-05-11T14:30:20,195 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_international_law.yaml' 2026-05-11T14:30:20,196 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_jurisprudence.yaml' 2026-05-11T14:30:20,197 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_logical_fallacies.yaml' 2026-05-11T14:30:20,199 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_machine_learning.yaml' 2026-05-11T14:30:20,200 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_management.yaml' 2026-05-11T14:30:20,201 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_marketing.yaml' 2026-05-11T14:30:20,202 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_medical_genetics.yaml' 2026-05-11T14:30:20,203 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_miscellaneous.yaml' 2026-05-11T14:30:20,205 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_disputes.yaml' 2026-05-11T14:30:20,206 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_scenarios.yaml' 2026-05-11T14:30:20,207 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_nutrition.yaml' 2026-05-11T14:30:20,208 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_philosophy.yaml' 2026-05-11T14:30:20,209 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_prehistory.yaml' 2026-05-11T14:30:20,211 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_accounting.yaml' 2026-05-11T14:30:20,212 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_law.yaml' 2026-05-11T14:30:20,213 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_medicine.yaml' 2026-05-11T14:30:20,214 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_psychology.yaml' 2026-05-11T14:30:20,216 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_public_relations.yaml' 2026-05-11T14:30:20,217 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_security_studies.yaml' 2026-05-11T14:30:20,218 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_sociology.yaml' 2026-05-11T14:30:20,219 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_us_foreign_policy.yaml' 2026-05-11T14:30:20,220 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_virology.yaml' 2026-05-11T14:30:20,222 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_world_religions.yaml' 2026-05-11T14:30:20,223 adding 'lm_eval/tasks/openai-mmmlu/default/utils.py' 2026-05-11T14:30:20,225 adding 'lm_eval/tasks/openbookqa/README.md' 2026-05-11T14:30:20,226 adding 'lm_eval/tasks/openbookqa/openbookqa.yaml' 2026-05-11T14:30:20,228 adding 'lm_eval/tasks/paloma/README.md' 2026-05-11T14:30:20,229 adding 'lm_eval/tasks/paloma/_paloma_template' 2026-05-11T14:30:20,230 adding 'lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml' 2026-05-11T14:30:20,231 adding 'lm_eval/tasks/paloma/paloma_c4_100_domains.yaml' 2026-05-11T14:30:20,232 adding 'lm_eval/tasks/paloma/paloma_c4_en.yaml' 2026-05-11T14:30:20,234 adding 'lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml' 2026-05-11T14:30:20,235 adding 'lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml' 2026-05-11T14:30:20,236 adding 'lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml' 2026-05-11T14:30:20,237 adding 'lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml' 2026-05-11T14:30:20,238 adding 'lm_eval/tasks/paloma/paloma_gab.yaml' 2026-05-11T14:30:20,239 adding 'lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml' 2026-05-11T14:30:20,240 adding 'lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml' 2026-05-11T14:30:20,242 adding 'lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml' 2026-05-11T14:30:20,243 adding 'lm_eval/tasks/paloma/paloma_mc4.yaml' 2026-05-11T14:30:20,244 adding 'lm_eval/tasks/paloma/paloma_ptb.yaml' 2026-05-11T14:30:20,245 adding 'lm_eval/tasks/paloma/paloma_redpajama.yaml' 2026-05-11T14:30:20,246 adding 'lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml' 2026-05-11T14:30:20,247 adding 'lm_eval/tasks/paloma/paloma_utils.py' 2026-05-11T14:30:20,248 adding 'lm_eval/tasks/paloma/paloma_wikitext_103.yaml' 2026-05-11T14:30:20,251 adding 'lm_eval/tasks/paws-x/README.md' 2026-05-11T14:30:20,252 adding 'lm_eval/tasks/paws-x/_generate_config.py' 2026-05-11T14:30:20,253 adding 'lm_eval/tasks/paws-x/_pawsx.yaml' 2026-05-11T14:30:20,254 adding 'lm_eval/tasks/paws-x/paws_de.yaml' 2026-05-11T14:30:20,256 adding 'lm_eval/tasks/paws-x/paws_en.yaml' 2026-05-11T14:30:20,257 adding 'lm_eval/tasks/paws-x/paws_es.yaml' 2026-05-11T14:30:20,258 adding 'lm_eval/tasks/paws-x/paws_fr.yaml' 2026-05-11T14:30:20,259 adding 'lm_eval/tasks/paws-x/paws_ja.yaml' 2026-05-11T14:30:20,260 adding 'lm_eval/tasks/paws-x/paws_ko.yaml' 2026-05-11T14:30:20,262 adding 'lm_eval/tasks/paws-x/paws_zh.yaml' 2026-05-11T14:30:20,263 adding 'lm_eval/tasks/paws-x/pawsx_template_yaml' 2026-05-11T14:30:20,264 adding 'lm_eval/tasks/paws-x/utils.py' 2026-05-11T14:30:20,266 adding 'lm_eval/tasks/pile/README.md' 2026-05-11T14:30:20,267 adding 'lm_eval/tasks/pile/pile_arxiv.yaml' 2026-05-11T14:30:20,269 adding 'lm_eval/tasks/pile/pile_bookcorpus2.yaml' 2026-05-11T14:30:20,270 adding 'lm_eval/tasks/pile/pile_books3.yaml' 2026-05-11T14:30:20,271 adding 'lm_eval/tasks/pile/pile_dm-mathematics.yaml' 2026-05-11T14:30:20,272 adding 'lm_eval/tasks/pile/pile_enron.yaml' 2026-05-11T14:30:20,273 adding 'lm_eval/tasks/pile/pile_europarl.yaml' 2026-05-11T14:30:20,274 adding 'lm_eval/tasks/pile/pile_freelaw.yaml' 2026-05-11T14:30:20,276 adding 'lm_eval/tasks/pile/pile_github.yaml' 2026-05-11T14:30:20,277 adding 'lm_eval/tasks/pile/pile_gutenberg.yaml' 2026-05-11T14:30:20,278 adding 'lm_eval/tasks/pile/pile_hackernews.yaml' 2026-05-11T14:30:20,279 adding 'lm_eval/tasks/pile/pile_nih-exporter.yaml' 2026-05-11T14:30:20,280 adding 'lm_eval/tasks/pile/pile_opensubtitles.yaml' 2026-05-11T14:30:20,281 adding 'lm_eval/tasks/pile/pile_openwebtext2.yaml' 2026-05-11T14:30:20,282 adding 'lm_eval/tasks/pile/pile_philpapers.yaml' 2026-05-11T14:30:20,283 adding 'lm_eval/tasks/pile/pile_pile-cc.yaml' 2026-05-11T14:30:20,284 adding 'lm_eval/tasks/pile/pile_pubmed-abstracts.yaml' 2026-05-11T14:30:20,286 adding 'lm_eval/tasks/pile/pile_pubmed-central.yaml' 2026-05-11T14:30:20,287 adding 'lm_eval/tasks/pile/pile_stackexchange.yaml' 2026-05-11T14:30:20,288 adding 'lm_eval/tasks/pile/pile_ubuntu-irc.yaml' 2026-05-11T14:30:20,289 adding 'lm_eval/tasks/pile/pile_uspto.yaml' 2026-05-11T14:30:20,290 adding 'lm_eval/tasks/pile/pile_wikipedia.yaml' 2026-05-11T14:30:20,291 adding 'lm_eval/tasks/pile/pile_youtubesubtitles.yaml' 2026-05-11T14:30:20,293 adding 'lm_eval/tasks/pile_10k/README.md' 2026-05-11T14:30:20,294 adding 'lm_eval/tasks/pile_10k/pile_10k.yaml' 2026-05-11T14:30:20,296 adding 'lm_eval/tasks/piqa/README.md' 2026-05-11T14:30:20,297 adding 'lm_eval/tasks/piqa/piqa.yaml' 2026-05-11T14:30:20,299 adding 'lm_eval/tasks/pisa/README.md' 2026-05-11T14:30:20,300 adding 'lm_eval/tasks/pisa/_pisa.yaml' 2026-05-11T14:30:20,302 adding 'lm_eval/tasks/pisa/_pisa_llm_judged.yaml' 2026-05-11T14:30:20,303 adding 'lm_eval/tasks/pisa/_template_yaml' 2026-05-11T14:30:20,304 adding 'lm_eval/tasks/pisa/pisa_ch.yaml' 2026-05-11T14:30:20,305 adding 'lm_eval/tasks/pisa/pisa_ch_llm_judged.yaml' 2026-05-11T14:30:20,306 adding 'lm_eval/tasks/pisa/pisa_de.yaml' 2026-05-11T14:30:20,307 adding 'lm_eval/tasks/pisa/pisa_de_llm_judged.yaml' 2026-05-11T14:30:20,308 adding 'lm_eval/tasks/pisa/pisa_en.yaml' 2026-05-11T14:30:20,309 adding 'lm_eval/tasks/pisa/pisa_en_llm_judged.yaml' 2026-05-11T14:30:20,310 adding 'lm_eval/tasks/pisa/pisa_es.yaml' 2026-05-11T14:30:20,311 adding 'lm_eval/tasks/pisa/pisa_es_llm_judged.yaml' 2026-05-11T14:30:20,313 adding 'lm_eval/tasks/pisa/pisa_fr.yaml' 2026-05-11T14:30:20,314 adding 'lm_eval/tasks/pisa/pisa_fr_llm_judged.yaml' 2026-05-11T14:30:20,315 adding 'lm_eval/tasks/pisa/pisa_it.yaml' 2026-05-11T14:30:20,316 adding 'lm_eval/tasks/pisa/pisa_it_llm_judged.yaml' 2026-05-11T14:30:20,318 adding 'lm_eval/tasks/pisa/utils.py' 2026-05-11T14:30:20,320 adding 'lm_eval/tasks/polemo2/README.md' 2026-05-11T14:30:20,321 adding 'lm_eval/tasks/polemo2/polemo2_in.yaml' 2026-05-11T14:30:20,322 adding 'lm_eval/tasks/polemo2/polemo2_out.yaml' 2026-05-11T14:30:20,324 adding 'lm_eval/tasks/portuguese_bench/README.md' 2026-05-11T14:30:20,325 adding 'lm_eval/tasks/portuguese_bench/assin_entailment.yaml' 2026-05-11T14:30:20,327 adding 'lm_eval/tasks/portuguese_bench/assin_paraphrase.yaml' 2026-05-11T14:30:20,328 adding 'lm_eval/tasks/portuguese_bench/portuguese_bench.yaml' 2026-05-11T14:30:20,330 adding 'lm_eval/tasks/portuguese_bench/flores_pt/_flores_common_yaml' 2026-05-11T14:30:20,331 adding 'lm_eval/tasks/portuguese_bench/flores_pt/create_yamls_flores_pt.py' 2026-05-11T14:30:20,332 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_ca-pt.yaml' 2026-05-11T14:30:20,334 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_de-pt.yaml' 2026-05-11T14:30:20,335 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_en-pt.yaml' 2026-05-11T14:30:20,336 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_es-pt.yaml' 2026-05-11T14:30:20,337 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_eu-pt.yaml' 2026-05-11T14:30:20,338 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_fr-pt.yaml' 2026-05-11T14:30:20,339 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_gl-pt.yaml' 2026-05-11T14:30:20,341 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_it-pt.yaml' 2026-05-11T14:30:20,342 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-ca.yaml' 2026-05-11T14:30:20,343 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-de.yaml' 2026-05-11T14:30:20,344 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-en.yaml' 2026-05-11T14:30:20,345 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-es.yaml' 2026-05-11T14:30:20,347 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-eu.yaml' 2026-05-11T14:30:20,348 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-fr.yaml' 2026-05-11T14:30:20,349 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-gl.yaml' 2026-05-11T14:30:20,350 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-it.yaml' 2026-05-11T14:30:20,351 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt.yaml' 2026-05-11T14:30:20,353 adding 'lm_eval/tasks/prost/README.md' 2026-05-11T14:30:20,354 adding 'lm_eval/tasks/prost/corypaik_prost.yaml' 2026-05-11T14:30:20,356 adding 'lm_eval/tasks/pubmedqa/README.md' 2026-05-11T14:30:20,357 adding 'lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py' 2026-05-11T14:30:20,358 adding 'lm_eval/tasks/pubmedqa/pubmedqa.yaml' 2026-05-11T14:30:20,360 adding 'lm_eval/tasks/qa4mre/README.md' 2026-05-11T14:30:20,361 adding 'lm_eval/tasks/qa4mre/preprocess_qa4mre.py' 2026-05-11T14:30:20,363 adding 'lm_eval/tasks/qa4mre/qa4mre_2011.yaml' 2026-05-11T14:30:20,364 adding 'lm_eval/tasks/qa4mre/qa4mre_2012.yaml' 2026-05-11T14:30:20,365 adding 'lm_eval/tasks/qa4mre/qa4mre_2013.yaml' 2026-05-11T14:30:20,367 adding 'lm_eval/tasks/qasper/README.md' 2026-05-11T14:30:20,368 adding 'lm_eval/tasks/qasper/bool.yaml' 2026-05-11T14:30:20,369 adding 'lm_eval/tasks/qasper/freeform.yaml' 2026-05-11T14:30:20,370 adding 'lm_eval/tasks/qasper/metrics.py' 2026-05-11T14:30:20,372 adding 'lm_eval/tasks/qasper/utils.py' 2026-05-11T14:30:20,374 adding 'lm_eval/tasks/race/README.md' 2026-05-11T14:30:20,375 adding 'lm_eval/tasks/race/preprocess_race.py' 2026-05-11T14:30:20,376 adding 'lm_eval/tasks/race/race.yaml' 2026-05-11T14:30:20,378 adding 'lm_eval/tasks/realtoxicityprompts/metric.py' 2026-05-11T14:30:20,379 adding 'lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml' 2026-05-11T14:30:20,382 adding 'lm_eval/tasks/ruler/README.md' 2026-05-11T14:30:20,383 adding 'lm_eval/tasks/ruler/common_utils.py' 2026-05-11T14:30:20,384 adding 'lm_eval/tasks/ruler/cwe.yaml' 2026-05-11T14:30:20,386 adding 'lm_eval/tasks/ruler/cwe_utils.py' 2026-05-11T14:30:20,387 adding 'lm_eval/tasks/ruler/essays.py' 2026-05-11T14:30:20,389 adding 'lm_eval/tasks/ruler/fwe.yaml' 2026-05-11T14:30:20,390 adding 'lm_eval/tasks/ruler/fwe_utils.py' 2026-05-11T14:30:20,391 adding 'lm_eval/tasks/ruler/niah_multikey_1.yaml' 2026-05-11T14:30:20,393 adding 'lm_eval/tasks/ruler/niah_multikey_2.yaml' 2026-05-11T14:30:20,394 adding 'lm_eval/tasks/ruler/niah_multikey_3.yaml' 2026-05-11T14:30:20,395 adding 'lm_eval/tasks/ruler/niah_multiquery.yaml' 2026-05-11T14:30:20,396 adding 'lm_eval/tasks/ruler/niah_multivalue.yaml' 2026-05-11T14:30:20,397 adding 'lm_eval/tasks/ruler/niah_single_1.yaml' 2026-05-11T14:30:20,399 adding 'lm_eval/tasks/ruler/niah_single_2.yaml' 2026-05-11T14:30:20,400 adding 'lm_eval/tasks/ruler/niah_single_3.yaml' 2026-05-11T14:30:20,401 adding 'lm_eval/tasks/ruler/niah_utils.py' 2026-05-11T14:30:20,403 adding 'lm_eval/tasks/ruler/prepare_niah.py' 2026-05-11T14:30:20,404 adding 'lm_eval/tasks/ruler/qa_hotpot.yaml' 2026-05-11T14:30:20,406 adding 'lm_eval/tasks/ruler/qa_squad.yaml' 2026-05-11T14:30:20,407 adding 'lm_eval/tasks/ruler/qa_utils.py' 2026-05-11T14:30:20,409 adding 'lm_eval/tasks/ruler/ruler.yaml' 2026-05-11T14:30:20,410 adding 'lm_eval/tasks/ruler/vt.yaml' 2026-05-11T14:30:20,412 adding 'lm_eval/tasks/ruler/vt_utils.py' 2026-05-11T14:30:20,414 adding 'lm_eval/tasks/sciq/README.md' 2026-05-11T14:30:20,415 adding 'lm_eval/tasks/sciq/sciq.yaml' 2026-05-11T14:30:20,417 adding 'lm_eval/tasks/score/NON_GREEDY.md' 2026-05-11T14:30:20,419 adding 'lm_eval/tasks/score/README.md' 2026-05-11T14:30:20,420 adding 'lm_eval/tasks/score/non_greedy.sh' 2026-05-11T14:30:20,422 adding 'lm_eval/tasks/score/non_greedy_summarizer.py' 2026-05-11T14:30:20,423 adding 'lm_eval/tasks/score/score_robustness.yaml' 2026-05-11T14:30:20,425 adding 'lm_eval/tasks/score/utils.py' 2026-05-11T14:30:20,427 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_aqua_rat.yaml' 2026-05-11T14:30:20,429 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_logiqa_en.yaml' 2026-05-11T14:30:20,430 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lsat_rc.yaml' 2026-05-11T14:30:20,431 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_ar.yaml' 2026-05-11T14:30:20,433 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_lr.yaml' 2026-05-11T14:30:20,434 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_en.yaml' 2026-05-11T14:30:20,435 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_math.yaml' 2026-05-11T14:30:20,436 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_aqua_rat.yaml' 2026-05-11T14:30:20,438 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_logiqa_en.yaml' 2026-05-11T14:30:20,439 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_ar.yaml' 2026-05-11T14:30:20,440 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_lr.yaml' 2026-05-11T14:30:20,441 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_rc.yaml' 2026-05-11T14:30:20,442 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_en.yaml' 2026-05-11T14:30:20,444 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_math.yaml' 2026-05-11T14:30:20,445 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_aqua_rat.yaml' 2026-05-11T14:30:20,446 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_logiqa_en.yaml' 2026-05-11T14:30:20,447 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lsat_rc.yaml' 2026-05-11T14:30:20,449 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_ar.yaml' 2026-05-11T14:30:20,450 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_lr.yaml' 2026-05-11T14:30:20,451 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_en.yaml' 2026-05-11T14:30:20,452 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_math.yaml' 2026-05-11T14:30:20,454 adding 'lm_eval/tasks/score/agi_eval/prompt_templates.json' 2026-05-11T14:30:20,455 adding 'lm_eval/tasks/score/agi_eval/score_non_greedy_robustness_agieval.yaml' 2026-05-11T14:30:20,456 adding 'lm_eval/tasks/score/agi_eval/score_option_order_robustness_agieval.yaml' 2026-05-11T14:30:20,458 adding 'lm_eval/tasks/score/agi_eval/score_prompt_robustness_agieval.yaml' 2026-05-11T14:30:20,459 adding 'lm_eval/tasks/score/agi_eval/score_robustness_agieval.yaml' 2026-05-11T14:30:20,460 adding 'lm_eval/tasks/score/agi_eval/utils_agieval.py' 2026-05-11T14:30:20,464 adding 'lm_eval/tasks/score/math/math_grader.py' 2026-05-11T14:30:20,466 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_algebra.yaml' 2026-05-11T14:30:20,467 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_counting_and_prob.yaml' 2026-05-11T14:30:20,468 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_geometry.yaml' 2026-05-11T14:30:20,469 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_intermediate_algebra.yaml' 2026-05-11T14:30:20,471 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_num_theory.yaml' 2026-05-11T14:30:20,472 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_prealgebra.yaml' 2026-05-11T14:30:20,473 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_precalc.yaml' 2026-05-11T14:30:20,474 adding 'lm_eval/tasks/score/math/prompt_robustness_math_algebra.yaml' 2026-05-11T14:30:20,476 adding 'lm_eval/tasks/score/math/prompt_robustness_math_counting_and_prob.yaml' 2026-05-11T14:30:20,477 adding 'lm_eval/tasks/score/math/prompt_robustness_math_geometry.yaml' 2026-05-11T14:30:20,478 adding 'lm_eval/tasks/score/math/prompt_robustness_math_intermediate_algebra.yaml' 2026-05-11T14:30:20,480 adding 'lm_eval/tasks/score/math/prompt_robustness_math_num_theory.yaml' 2026-05-11T14:30:20,481 adding 'lm_eval/tasks/score/math/prompt_robustness_math_prealgebra.yaml' 2026-05-11T14:30:20,482 adding 'lm_eval/tasks/score/math/prompt_robustness_math_precalc.yaml' 2026-05-11T14:30:20,483 adding 'lm_eval/tasks/score/math/prompt_templates.json' 2026-05-11T14:30:20,485 adding 'lm_eval/tasks/score/math/score_non_greedy_robustness_math.yaml' 2026-05-11T14:30:20,486 adding 'lm_eval/tasks/score/math/score_prompt_robustness_math.yaml' 2026-05-11T14:30:20,487 adding 'lm_eval/tasks/score/math/score_robustness_math.yaml' 2026-05-11T14:30:20,488 adding 'lm_eval/tasks/score/math/to_be_fixed_questions.json' 2026-05-11T14:30:20,490 adding 'lm_eval/tasks/score/math/utils_math.py' 2026-05-11T14:30:20,492 adding 'lm_eval/tasks/score/mmlu_pro/prompt_templates.json' 2026-05-11T14:30:20,493 adding 'lm_eval/tasks/score/mmlu_pro/score_non_greedy_robustness_mmlu_pro.yaml' 2026-05-11T14:30:20,494 adding 'lm_eval/tasks/score/mmlu_pro/score_option_order_robustness_mmlu_pro.yaml' 2026-05-11T14:30:20,496 adding 'lm_eval/tasks/score/mmlu_pro/score_prompt_robustness_mmlu_pro.yaml' 2026-05-11T14:30:20,497 adding 'lm_eval/tasks/score/mmlu_pro/utils_mmlu_pro.py' 2026-05-11T14:30:20,500 adding 'lm_eval/tasks/scrolls/README.md' 2026-05-11T14:30:20,501 adding 'lm_eval/tasks/scrolls/scrolls_contractnli.yaml' 2026-05-11T14:30:20,502 adding 'lm_eval/tasks/scrolls/scrolls_govreport.yaml' 2026-05-11T14:30:20,503 adding 'lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml' 2026-05-11T14:30:20,504 adding 'lm_eval/tasks/scrolls/scrolls_qasper.yaml' 2026-05-11T14:30:20,505 adding 'lm_eval/tasks/scrolls/scrolls_qmsum.yaml' 2026-05-11T14:30:20,506 adding 'lm_eval/tasks/scrolls/scrolls_quality.yaml' 2026-05-11T14:30:20,507 adding 'lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml' 2026-05-11T14:30:20,510 adding 'lm_eval/tasks/scrolls/task.py' 2026-05-11T14:30:20,512 adding 'lm_eval/tasks/simple_cooccurrence_bias/README.md' 2026-05-11T14:30:20,513 adding 'lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias.yaml' 2026-05-11T14:30:20,514 adding 'lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias_gen.yaml' 2026-05-11T14:30:20,515 adding 'lm_eval/tasks/simple_cooccurrence_bias/utils.py' 2026-05-11T14:30:20,517 adding 'lm_eval/tasks/siqa/README.md' 2026-05-11T14:30:20,519 adding 'lm_eval/tasks/siqa/siqa.yaml' 2026-05-11T14:30:20,520 adding 'lm_eval/tasks/slr_bench/README.md' 2026-05-11T14:30:20,522 adding 'lm_eval/tasks/slr_bench/lm_eval_slr_bench.py' 2026-05-11T14:30:20,523 adding 'lm_eval/tasks/slr_bench/slr_bench_all.yaml' 2026-05-11T14:30:20,524 adding 'lm_eval/tasks/slr_bench/slr_bench_basic.yaml' 2026-05-11T14:30:20,525 adding 'lm_eval/tasks/slr_bench/slr_bench_common_yaml' 2026-05-11T14:30:20,527 adding 'lm_eval/tasks/slr_bench/slr_bench_easy.yaml' 2026-05-11T14:30:20,528 adding 'lm_eval/tasks/slr_bench/slr_bench_group.yaml' 2026-05-11T14:30:20,529 adding 'lm_eval/tasks/slr_bench/slr_bench_hard.yaml' 2026-05-11T14:30:20,530 adding 'lm_eval/tasks/slr_bench/slr_bench_medium.yaml' 2026-05-11T14:30:20,533 adding 'lm_eval/tasks/spanish_bench/README.md' 2026-05-11T14:30:20,534 adding 'lm_eval/tasks/spanish_bench/cocoteros_es.yaml' 2026-05-11T14:30:20,535 adding 'lm_eval/tasks/spanish_bench/copa_es.yaml' 2026-05-11T14:30:20,536 adding 'lm_eval/tasks/spanish_bench/escola.yaml' 2026-05-11T14:30:20,537 adding 'lm_eval/tasks/spanish_bench/mgsm_direct_es_spanish_bench.yaml' 2026-05-11T14:30:20,539 adding 'lm_eval/tasks/spanish_bench/openbookqa_es.yaml' 2026-05-11T14:30:20,540 adding 'lm_eval/tasks/spanish_bench/paws_es_spanish_bench.yaml' 2026-05-11T14:30:20,541 adding 'lm_eval/tasks/spanish_bench/spanish_bench.yaml' 2026-05-11T14:30:20,542 adding 'lm_eval/tasks/spanish_bench/utils.py' 2026-05-11T14:30:20,544 adding 'lm_eval/tasks/spanish_bench/wnli_es.yaml' 2026-05-11T14:30:20,545 adding 'lm_eval/tasks/spanish_bench/xlsum_es.yaml' 2026-05-11T14:30:20,546 adding 'lm_eval/tasks/spanish_bench/xnli_es_spanish_bench.yaml' 2026-05-11T14:30:20,549 adding 'lm_eval/tasks/spanish_bench/flores_es/_flores_common_yaml' 2026-05-11T14:30:20,550 adding 'lm_eval/tasks/spanish_bench/flores_es/create_yamls_flores_es.py' 2026-05-11T14:30:20,551 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_ca-es.yaml' 2026-05-11T14:30:20,552 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_de-es.yaml' 2026-05-11T14:30:20,553 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_en-es.yaml' 2026-05-11T14:30:20,554 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-ca.yaml' 2026-05-11T14:30:20,555 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-de.yaml' 2026-05-11T14:30:20,557 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-en.yaml' 2026-05-11T14:30:20,558 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-eu.yaml' 2026-05-11T14:30:20,559 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-fr.yaml' 2026-05-11T14:30:20,560 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-gl.yaml' 2026-05-11T14:30:20,561 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-it.yaml' 2026-05-11T14:30:20,562 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-pt.yaml' 2026-05-11T14:30:20,563 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es.yaml' 2026-05-11T14:30:20,565 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_eu-es.yaml' 2026-05-11T14:30:20,566 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_fr-es.yaml' 2026-05-11T14:30:20,567 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_gl-es.yaml' 2026-05-11T14:30:20,568 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_it-es.yaml' 2026-05-11T14:30:20,569 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_pt-es.yaml' 2026-05-11T14:30:20,571 adding 'lm_eval/tasks/spanish_bench/phrases_es/_phrases_es_common' 2026-05-11T14:30:20,572 adding 'lm_eval/tasks/spanish_bench/phrases_es/phrases_es-va.yaml' 2026-05-11T14:30:20,573 adding 'lm_eval/tasks/spanish_bench/phrases_es/phrases_va-es.yaml' 2026-05-11T14:30:20,575 adding 'lm_eval/tasks/squad_completion/README.md' 2026-05-11T14:30:20,576 adding 'lm_eval/tasks/squad_completion/squad_completion.yaml' 2026-05-11T14:30:20,577 adding 'lm_eval/tasks/squad_completion/task.py' 2026-05-11T14:30:20,579 adding 'lm_eval/tasks/squadv2/README.md' 2026-05-11T14:30:20,580 adding 'lm_eval/tasks/squadv2/squadv2.yaml' 2026-05-11T14:30:20,582 adding 'lm_eval/tasks/squadv2/task.py' 2026-05-11T14:30:20,584 adding 'lm_eval/tasks/storycloze/README.md' 2026-05-11T14:30:20,585 adding 'lm_eval/tasks/storycloze/storycloze_2016.yaml' 2026-05-11T14:30:20,586 adding 'lm_eval/tasks/storycloze/storycloze_2018.yaml' 2026-05-11T14:30:20,588 adding 'lm_eval/tasks/super_glue/README.md' 2026-05-11T14:30:20,590 adding 'lm_eval/tasks/super_glue/boolq/default.yaml' 2026-05-11T14:30:20,591 adding 'lm_eval/tasks/super_glue/boolq/seq2seq.yaml' 2026-05-11T14:30:20,592 adding 'lm_eval/tasks/super_glue/boolq/t5-prompt.yaml' 2026-05-11T14:30:20,594 adding 'lm_eval/tasks/super_glue/cb/aggregate.py' 2026-05-11T14:30:20,595 adding 'lm_eval/tasks/super_glue/cb/default.yaml' 2026-05-11T14:30:20,596 adding 'lm_eval/tasks/super_glue/cb/t5-prompt.yaml' 2026-05-11T14:30:20,598 adding 'lm_eval/tasks/super_glue/cb/t5_utils.py' 2026-05-11T14:30:20,599 adding 'lm_eval/tasks/super_glue/copa/default.yaml' 2026-05-11T14:30:20,601 adding 'lm_eval/tasks/super_glue/copa/t5-prompt.yaml' 2026-05-11T14:30:20,602 adding 'lm_eval/tasks/super_glue/copa/utils.py' 2026-05-11T14:30:20,604 adding 'lm_eval/tasks/super_glue/multirc/default.yaml' 2026-05-11T14:30:20,605 adding 'lm_eval/tasks/super_glue/multirc/t5-prompt.yaml' 2026-05-11T14:30:20,606 adding 'lm_eval/tasks/super_glue/multirc/t5_utils.py' 2026-05-11T14:30:20,608 adding 'lm_eval/tasks/super_glue/record/default.yaml' 2026-05-11T14:30:20,609 adding 'lm_eval/tasks/super_glue/record/t5-prompt.yaml' 2026-05-11T14:30:20,611 adding 'lm_eval/tasks/super_glue/record/t5_utils.py' 2026-05-11T14:30:20,612 adding 'lm_eval/tasks/super_glue/record/util.py' 2026-05-11T14:30:20,614 adding 'lm_eval/tasks/super_glue/rte/default.yaml' 2026-05-11T14:30:20,615 adding 'lm_eval/tasks/super_glue/rte/t5-prompt.yaml' 2026-05-11T14:30:20,617 adding 'lm_eval/tasks/super_glue/wic/default.yaml' 2026-05-11T14:30:20,618 adding 'lm_eval/tasks/super_glue/wic/t5-prompt.yaml' 2026-05-11T14:30:20,619 adding 'lm_eval/tasks/super_glue/wsc/default.yaml' 2026-05-11T14:30:20,621 adding 'lm_eval/tasks/super_glue/wsc/preprocess_wsc.py' 2026-05-11T14:30:20,622 adding 'lm_eval/tasks/super_glue/wsc/t5-prompt.yaml' 2026-05-11T14:30:20,623 adding 'lm_eval/tasks/super_glue/wsc/t5_utils.py' 2026-05-11T14:30:20,625 adding 'lm_eval/tasks/swag/README.md' 2026-05-11T14:30:20,626 adding 'lm_eval/tasks/swag/swag.yaml' 2026-05-11T14:30:20,628 adding 'lm_eval/tasks/swde/README.md' 2026-05-11T14:30:20,629 adding 'lm_eval/tasks/swde/swde.yaml' 2026-05-11T14:30:20,630 adding 'lm_eval/tasks/swde/task.py' 2026-05-11T14:30:20,632 adding 'lm_eval/tasks/tinyBenchmarks/README.md' 2026-05-11T14:30:20,633 adding 'lm_eval/tasks/tinyBenchmarks/agg_functions.py' 2026-05-11T14:30:20,635 adding 'lm_eval/tasks/tinyBenchmarks/tinyArc.yaml' 2026-05-11T14:30:20,636 adding 'lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml' 2026-05-11T14:30:20,637 adding 'lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml' 2026-05-11T14:30:20,638 adding 'lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml' 2026-05-11T14:30:20,639 adding 'lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml' 2026-05-11T14:30:20,640 adding 'lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml' 2026-05-11T14:30:20,641 adding 'lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml' 2026-05-11T14:30:20,643 adding 'lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml' 2026-05-11T14:30:20,644 adding 'lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py' 2026-05-11T14:30:20,645 adding 'lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py' 2026-05-11T14:30:20,647 adding 'lm_eval/tasks/tinyBenchmarks/utils_winogrande.py' 2026-05-11T14:30:20,649 adding 'lm_eval/tasks/tmlu/README.md' 2026-05-11T14:30:20,650 adding 'lm_eval/tasks/tmlu/subject.tsv' 2026-05-11T14:30:20,652 adding 'lm_eval/tasks/tmlu/default/_default_template_yaml' 2026-05-11T14:30:20,654 adding 'lm_eval/tasks/tmlu/default/_generate_configs.py' 2026-05-11T14:30:20,655 adding 'lm_eval/tasks/tmlu/default/_tmlu.yaml' 2026-05-11T14:30:20,656 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_biology.yaml' 2026-05-11T14:30:20,657 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_chemistry.yaml' 2026-05-11T14:30:20,659 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_chinese.yaml' 2026-05-11T14:30:20,660 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_civics.yaml' 2026-05-11T14:30:20,661 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_geography.yaml' 2026-05-11T14:30:20,662 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_history.yaml' 2026-05-11T14:30:20,664 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_biology.yaml' 2026-05-11T14:30:20,665 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_chemistry.yaml' 2026-05-11T14:30:20,666 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_chinese.yaml' 2026-05-11T14:30:20,667 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_civics.yaml' 2026-05-11T14:30:20,669 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_earth_science.yaml' 2026-05-11T14:30:20,670 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_geography.yaml' 2026-05-11T14:30:20,671 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_history.yaml' 2026-05-11T14:30:20,672 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_biology.yaml' 2026-05-11T14:30:20,674 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_chemistry.yaml' 2026-05-11T14:30:20,675 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_chinese.yaml' 2026-05-11T14:30:20,676 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_civics.yaml' 2026-05-11T14:30:20,677 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_earth_science.yaml' 2026-05-11T14:30:20,679 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_geography.yaml' 2026-05-11T14:30:20,680 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_history.yaml' 2026-05-11T14:30:20,681 adding 'lm_eval/tasks/tmlu/default/tmlu_accountant.yaml' 2026-05-11T14:30:20,682 adding 'lm_eval/tasks/tmlu/default/tmlu_basic_traditional_chinese_medicine.yaml' 2026-05-11T14:30:20,684 adding 'lm_eval/tasks/tmlu/default/tmlu_clinical_psychologist.yaml' 2026-05-11T14:30:20,685 adding 'lm_eval/tasks/tmlu/default/tmlu_clinical_traditional_chinese_medicine.yaml' 2026-05-11T14:30:20,686 adding 'lm_eval/tasks/tmlu/default/tmlu_driving_rule.yaml' 2026-05-11T14:30:20,687 adding 'lm_eval/tasks/tmlu/default/tmlu_lawyer_qualification.yaml' 2026-05-11T14:30:20,688 adding 'lm_eval/tasks/tmlu/default/tmlu_nutritionist.yaml' 2026-05-11T14:30:20,689 adding 'lm_eval/tasks/tmlu/default/tmlu_taiwan_tourist_resources.yaml' 2026-05-11T14:30:20,691 adding 'lm_eval/tasks/tmlu/default/tmlu_teacher_qualification.yaml' 2026-05-11T14:30:20,692 adding 'lm_eval/tasks/tmlu/default/tmlu_tour_guide.yaml' 2026-05-11T14:30:20,693 adding 'lm_eval/tasks/tmlu/default/tmlu_tour_leader.yaml' 2026-05-11T14:30:20,694 adding 'lm_eval/tasks/tmlu/default/utils.py' 2026-05-11T14:30:20,696 adding 'lm_eval/tasks/tmmluplus/README.md' 2026-05-11T14:30:20,697 adding 'lm_eval/tasks/tmmluplus/subject.tsv' 2026-05-11T14:30:20,701 adding 'lm_eval/tasks/tmmluplus/default/_generate_configs.py' 2026-05-11T14:30:20,702 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus.yaml' 2026-05-11T14:30:20,703 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_STEM.yaml' 2026-05-11T14:30:20,704 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_humanities.yaml' 2026-05-11T14:30:20,705 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_other.yaml' 2026-05-11T14:30:20,706 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_social_sciences.yaml' 2026-05-11T14:30:20,708 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_template_yaml' 2026-05-11T14:30:20,709 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml' 2026-05-11T14:30:20,710 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml' 2026-05-11T14:30:20,711 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml' 2026-05-11T14:30:20,712 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml' 2026-05-11T14:30:20,713 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml' 2026-05-11T14:30:20,715 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml' 2026-05-11T14:30:20,716 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml' 2026-05-11T14:30:20,717 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml' 2026-05-11T14:30:20,718 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml' 2026-05-11T14:30:20,720 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml' 2026-05-11T14:30:20,721 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml' 2026-05-11T14:30:20,722 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml' 2026-05-11T14:30:20,723 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml' 2026-05-11T14:30:20,724 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml' 2026-05-11T14:30:20,726 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml' 2026-05-11T14:30:20,727 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml' 2026-05-11T14:30:20,728 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml' 2026-05-11T14:30:20,729 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml' 2026-05-11T14:30:20,730 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml' 2026-05-11T14:30:20,732 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml' 2026-05-11T14:30:20,733 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml' 2026-05-11T14:30:20,734 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml' 2026-05-11T14:30:20,735 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml' 2026-05-11T14:30:20,737 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml' 2026-05-11T14:30:20,738 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml' 2026-05-11T14:30:20,739 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml' 2026-05-11T14:30:20,740 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml' 2026-05-11T14:30:20,741 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml' 2026-05-11T14:30:20,742 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml' 2026-05-11T14:30:20,743 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml' 2026-05-11T14:30:20,744 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml' 2026-05-11T14:30:20,746 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml' 2026-05-11T14:30:20,747 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml' 2026-05-11T14:30:20,748 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml' 2026-05-11T14:30:20,749 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml' 2026-05-11T14:30:20,751 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml' 2026-05-11T14:30:20,752 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml' 2026-05-11T14:30:20,753 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml' 2026-05-11T14:30:20,754 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml' 2026-05-11T14:30:20,755 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml' 2026-05-11T14:30:20,756 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml' 2026-05-11T14:30:20,757 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml' 2026-05-11T14:30:20,759 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml' 2026-05-11T14:30:20,760 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml' 2026-05-11T14:30:20,761 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml' 2026-05-11T14:30:20,762 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml' 2026-05-11T14:30:20,763 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml' 2026-05-11T14:30:20,765 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml' 2026-05-11T14:30:20,766 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml' 2026-05-11T14:30:20,767 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml' 2026-05-11T14:30:20,768 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml' 2026-05-11T14:30:20,769 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml' 2026-05-11T14:30:20,771 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml' 2026-05-11T14:30:20,772 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml' 2026-05-11T14:30:20,773 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml' 2026-05-11T14:30:20,774 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml' 2026-05-11T14:30:20,775 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml' 2026-05-11T14:30:20,777 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml' 2026-05-11T14:30:20,778 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml' 2026-05-11T14:30:20,779 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml' 2026-05-11T14:30:20,780 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml' 2026-05-11T14:30:20,781 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml' 2026-05-11T14:30:20,783 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml' 2026-05-11T14:30:20,784 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml' 2026-05-11T14:30:20,785 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml' 2026-05-11T14:30:20,786 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml' 2026-05-11T14:30:20,787 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml' 2026-05-11T14:30:20,788 adding 'lm_eval/tasks/tmmluplus/default/utils.py' 2026-05-11T14:30:20,791 adding 'lm_eval/tasks/toksuite/README.md' 2026-05-11T14:30:20,792 adding 'lm_eval/tasks/toksuite/_toksuite_abridged.yaml' 2026-05-11T14:30:20,793 adding 'lm_eval/tasks/toksuite/_toksuite_all.yaml' 2026-05-11T14:30:20,796 adding 'lm_eval/tasks/toksuite/utils.py' 2026-05-11T14:30:20,798 adding 'lm_eval/tasks/toksuite/toksuite_chinese/_toksuite_chinese.yaml' 2026-05-11T14:30:20,799 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_base.yaml' 2026-05-11T14:30:20,800 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_canonical.yaml' 2026-05-11T14:30:20,801 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_code_language_script_switching.yaml' 2026-05-11T14:30:20,802 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_colloquial.yaml' 2026-05-11T14:30:20,803 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_equivalent_expressions.yaml' 2026-05-11T14:30:20,805 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_keyboard_proximity_errors.yaml' 2026-05-11T14:30:20,806 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_ocr_errors.yaml' 2026-05-11T14:30:20,807 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_optional_diacritics.yaml' 2026-05-11T14:30:20,808 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_partially_romanized.yaml' 2026-05-11T14:30:20,809 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_romanization.yaml' 2026-05-11T14:30:20,811 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_space_removal.yaml' 2026-05-11T14:30:20,812 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_spelled_out.yaml' 2026-05-11T14:30:20,813 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_traditional.yaml' 2026-05-11T14:30:20,814 adding 'lm_eval/tasks/toksuite/toksuite_chinese/toksuite_chinese_word_spacing_zero-width_characters_extra_space.yaml' 2026-05-11T14:30:20,817 adding 'lm_eval/tasks/toksuite/toksuite_english/_toksuite_english.yaml' 2026-05-11T14:30:20,819 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_abbreviations.yaml' 2026-05-11T14:30:20,821 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_base.yaml' 2026-05-11T14:30:20,822 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_canonical.yaml' 2026-05-11T14:30:20,823 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_capitalization.yaml' 2026-05-11T14:30:20,824 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_character_deletion.yaml' 2026-05-11T14:30:20,826 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_character_substitution.yaml' 2026-05-11T14:30:20,827 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_colloquial.yaml' 2026-05-11T14:30:20,828 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_compounds.yaml' 2026-05-11T14:30:20,829 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_contractions.yaml' 2026-05-11T14:30:20,830 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_date_formats.yaml' 2026-05-11T14:30:20,831 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_emoji_substitution.yaml' 2026-05-11T14:30:20,833 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_grammatical_errors.yaml' 2026-05-11T14:30:20,834 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_historical_spelling.yaml' 2026-05-11T14:30:20,835 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_homoglyphs.yaml' 2026-05-11T14:30:20,836 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_hyphenated_spelling.yaml' 2026-05-11T14:30:20,838 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_inflections.yaml' 2026-05-11T14:30:20,839 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_keyboard_proximity_errors.yaml' 2026-05-11T14:30:20,840 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_letter_repetition_for_emphasis.yaml' 2026-05-11T14:30:20,841 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_lowercase.yaml' 2026-05-11T14:30:20,842 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_macron_diacritic.yaml' 2026-05-11T14:30:20,843 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_ocr_errors.yaml' 2026-05-11T14:30:20,844 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_orthographic_errors.yaml' 2026-05-11T14:30:20,846 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_scripted_text.yaml' 2026-05-11T14:30:20,847 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_similar_words.yaml' 2026-05-11T14:30:20,848 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_space_removal.yaml' 2026-05-11T14:30:20,849 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_spaced_styling.yaml' 2026-05-11T14:30:20,850 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_spelled_out.yaml' 2026-05-11T14:30:20,851 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_superscript_subscript_styling.yaml' 2026-05-11T14:30:20,853 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_web_search_query.yaml' 2026-05-11T14:30:20,854 adding 'lm_eval/tasks/toksuite/toksuite_english/toksuite_english_word_reordering.yaml' 2026-05-11T14:30:20,856 adding 'lm_eval/tasks/toksuite/toksuite_farsi/_toksuite_farsi.yaml' 2026-05-11T14:30:20,857 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_arabic_keyboard_for_farsi.yaml' 2026-05-11T14:30:20,858 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_base.yaml' 2026-05-11T14:30:20,859 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_canonical.yaml' 2026-05-11T14:30:20,861 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_code_language_script_switching.yaml' 2026-05-11T14:30:20,862 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_colloquial.yaml' 2026-05-11T14:30:20,863 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_dialects.yaml' 2026-05-11T14:30:20,864 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_equivalent_expressions.yaml' 2026-05-11T14:30:20,865 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_keyboard_proximity_errors.yaml' 2026-05-11T14:30:20,866 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_number_romanization.yaml' 2026-05-11T14:30:20,868 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_optional_diacritics.yaml' 2026-05-11T14:30:20,869 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_romanization.yaml' 2026-05-11T14:30:20,870 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_spelled_out.yaml' 2026-05-11T14:30:20,871 adding 'lm_eval/tasks/toksuite/toksuite_farsi/toksuite_farsi_word_spacing_zero-width_characters_extra_space.yaml' 2026-05-11T14:30:20,873 adding 'lm_eval/tasks/toksuite/toksuite_italian/_toksuite_italian.yaml' 2026-05-11T14:30:20,874 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_abbreviations.yaml' 2026-05-11T14:30:20,875 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_base.yaml' 2026-05-11T14:30:20,877 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_canonical.yaml' 2026-05-11T14:30:20,878 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_capitalization.yaml' 2026-05-11T14:30:20,879 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_code_language_script_switching.yaml' 2026-05-11T14:30:20,880 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_contractions.yaml' 2026-05-11T14:30:20,881 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_date_formats.yaml' 2026-05-11T14:30:20,883 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_dialects.yaml' 2026-05-11T14:30:20,884 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_english_keyboard.yaml' 2026-05-11T14:30:20,885 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_grammatical_errors.yaml' 2026-05-11T14:30:20,886 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_keyboard_proximity_errors.yaml' 2026-05-11T14:30:20,887 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_numerical_formats.yaml' 2026-05-11T14:30:20,888 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_orthographic_errors.yaml' 2026-05-11T14:30:20,889 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_phonetic_spelling.yaml' 2026-05-11T14:30:20,890 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_plausible_diacritics_errors.yaml' 2026-05-11T14:30:20,891 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_similar_words.yaml' 2026-05-11T14:30:20,892 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_spelled_out.yaml' 2026-05-11T14:30:20,894 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_typographical_errors.yaml' 2026-05-11T14:30:20,895 adding 'lm_eval/tasks/toksuite/toksuite_italian/toksuite_italian_web_search_query.yaml' 2026-05-11T14:30:20,896 adding 'lm_eval/tasks/toksuite/toksuite_math/_toksuite_math.yaml' 2026-05-11T14:30:20,898 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_base.yaml' 2026-05-11T14:30:20,899 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_canonical.yaml' 2026-05-11T14:30:20,900 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_chinese.yaml' 2026-05-11T14:30:20,901 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_decorative_unicode.yaml' 2026-05-11T14:30:20,902 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_farsi.yaml' 2026-05-11T14:30:20,903 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_italian.yaml' 2026-05-11T14:30:20,904 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_latex.yaml' 2026-05-11T14:30:20,906 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_space_removal.yaml' 2026-05-11T14:30:20,907 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_spelled_out.yaml' 2026-05-11T14:30:20,908 adding 'lm_eval/tasks/toksuite/toksuite_math/toksuite_math_turkish.yaml' 2026-05-11T14:30:20,910 adding 'lm_eval/tasks/toksuite/toksuite_stem/_toksuite_stem.yaml' 2026-05-11T14:30:20,912 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_base.yaml' 2026-05-11T14:30:20,913 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_canonical.yaml' 2026-05-11T14:30:20,914 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_character_deletion.yaml' 2026-05-11T14:30:20,915 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_colloquial.yaml' 2026-05-11T14:30:20,916 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_diacriticized_styling.yaml' 2026-05-11T14:30:20,917 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_double_struck.yaml' 2026-05-11T14:30:20,919 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_enclosed_characters.yaml' 2026-05-11T14:30:20,920 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_equivalent_expressions.yaml' 2026-05-11T14:30:20,921 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_fullwidth_characters.yaml' 2026-05-11T14:30:20,922 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_latex.yaml' 2026-05-11T14:30:20,924 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_morpheme_separation.yaml' 2026-05-11T14:30:20,925 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_scripted_text.yaml' 2026-05-11T14:30:20,926 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_space_removal.yaml' 2026-05-11T14:30:20,927 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_spelled_out.yaml' 2026-05-11T14:30:20,928 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_strikethrough.yaml' 2026-05-11T14:30:20,929 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_superscript_subscript.yaml' 2026-05-11T14:30:20,930 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_typographical_errors.yaml' 2026-05-11T14:30:20,932 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_unicode_formatting.yaml' 2026-05-11T14:30:20,933 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_unusual_formatting.yaml' 2026-05-11T14:30:20,934 adding 'lm_eval/tasks/toksuite/toksuite_stem/toksuite_stem_upside_down_rotated.yaml' 2026-05-11T14:30:20,935 adding 'lm_eval/tasks/toksuite/toksuite_turkish/_toksuite_turkish.yaml' 2026-05-11T14:30:20,937 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_base.yaml' 2026-05-11T14:30:20,938 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_canonical.yaml' 2026-05-11T14:30:20,939 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_code_language_script_switching.yaml' 2026-05-11T14:30:20,940 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_colloquial.yaml' 2026-05-11T14:30:20,942 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_date_formats.yaml' 2026-05-11T14:30:20,943 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_derivations.yaml' 2026-05-11T14:30:20,944 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_dialects.yaml' 2026-05-11T14:30:20,945 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_english_keyboard.yaml' 2026-05-11T14:30:20,946 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_equivalent_expressions.yaml' 2026-05-11T14:30:20,947 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_grammatical_errors.yaml' 2026-05-11T14:30:20,948 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_inflections.yaml' 2026-05-11T14:30:20,949 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_keyboard_proximity_errors.yaml' 2026-05-11T14:30:20,951 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_orthographic_errors.yaml' 2026-05-11T14:30:20,952 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_similar_words.yaml' 2026-05-11T14:30:20,953 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_spelled_out.yaml' 2026-05-11T14:30:20,954 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_typographical_errors.yaml' 2026-05-11T14:30:20,955 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_web_search_query.yaml' 2026-05-11T14:30:20,956 adding 'lm_eval/tasks/toksuite/toksuite_turkish/toksuite_turkish_word_reordering.yaml' 2026-05-11T14:30:20,958 adding 'lm_eval/tasks/toxigen/README.md' 2026-05-11T14:30:20,959 adding 'lm_eval/tasks/toxigen/toxigen.yaml' 2026-05-11T14:30:20,961 adding 'lm_eval/tasks/toxigen/utils.py' 2026-05-11T14:30:20,962 adding 'lm_eval/tasks/translation/README.md' 2026-05-11T14:30:20,964 adding 'lm_eval/tasks/translation/iwslt2017_ar-en.yaml' 2026-05-11T14:30:20,965 adding 'lm_eval/tasks/translation/iwslt2017_en-ar.yaml' 2026-05-11T14:30:20,966 adding 'lm_eval/tasks/translation/utils.py' 2026-05-11T14:30:20,968 adding 'lm_eval/tasks/translation/wmt14_en-fr.yaml' 2026-05-11T14:30:20,969 adding 'lm_eval/tasks/translation/wmt14_fr-en.yaml' 2026-05-11T14:30:20,970 adding 'lm_eval/tasks/translation/wmt16_de-en.yaml' 2026-05-11T14:30:20,971 adding 'lm_eval/tasks/translation/wmt16_en-de.yaml' 2026-05-11T14:30:20,972 adding 'lm_eval/tasks/translation/wmt16_en-ro.yaml' 2026-05-11T14:30:20,973 adding 'lm_eval/tasks/translation/wmt16_ro-en.yaml' 2026-05-11T14:30:20,975 adding 'lm_eval/tasks/translation/wmt_common_yaml' 2026-05-11T14:30:20,977 adding 'lm_eval/tasks/triviaqa/README.md' 2026-05-11T14:30:20,978 adding 'lm_eval/tasks/triviaqa/default.yaml' 2026-05-11T14:30:20,980 adding 'lm_eval/tasks/truthfulqa/README.md' 2026-05-11T14:30:20,981 adding 'lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml' 2026-05-11T14:30:20,982 adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml' 2026-05-11T14:30:20,983 adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml' 2026-05-11T14:30:20,985 adding 'lm_eval/tasks/truthfulqa/utils.py' 2026-05-11T14:30:20,987 adding 'lm_eval/tasks/truthfulqa-multi/README.md' 2026-05-11T14:30:20,988 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_ca.yaml' 2026-05-11T14:30:20,989 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_common' 2026-05-11T14:30:20,990 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_en.yaml' 2026-05-11T14:30:20,991 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_es.yaml' 2026-05-11T14:30:20,992 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_eu.yaml' 2026-05-11T14:30:20,993 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_gl.yaml' 2026-05-11T14:30:20,995 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_ca.yaml' 2026-05-11T14:30:20,996 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_en.yaml' 2026-05-11T14:30:20,997 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_es.yaml' 2026-05-11T14:30:20,998 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_eu.yaml' 2026-05-11T14:30:20,999 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_gl.yaml' 2026-05-11T14:30:21,001 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_ca.yaml' 2026-05-11T14:30:21,002 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_en.yaml' 2026-05-11T14:30:21,003 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_es.yaml' 2026-05-11T14:30:21,004 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_eu.yaml' 2026-05-11T14:30:21,005 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_gl.yaml' 2026-05-11T14:30:21,007 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc_common' 2026-05-11T14:30:21,008 adding 'lm_eval/tasks/truthfulqa-multi/utils.py' 2026-05-11T14:30:21,010 adding 'lm_eval/tasks/turblimp/README.md' 2026-05-11T14:30:21,012 adding 'lm_eval/tasks/turblimp/_template_yaml' 2026-05-11T14:30:21,013 adding 'lm_eval/tasks/turblimp/anaphor_agreement.yaml' 2026-05-11T14:30:21,014 adding 'lm_eval/tasks/turblimp/argument_structure_ditransitive.yaml' 2026-05-11T14:30:21,015 adding 'lm_eval/tasks/turblimp/argument_structure_transitive.yaml' 2026-05-11T14:30:21,016 adding 'lm_eval/tasks/turblimp/binding.yaml' 2026-05-11T14:30:21,017 adding 'lm_eval/tasks/turblimp/determiners.yaml' 2026-05-11T14:30:21,018 adding 'lm_eval/tasks/turblimp/ellipsis.yaml' 2026-05-11T14:30:21,020 adding 'lm_eval/tasks/turblimp/irregular_forms.yaml' 2026-05-11T14:30:21,021 adding 'lm_eval/tasks/turblimp/island_effects.yaml' 2026-05-11T14:30:21,022 adding 'lm_eval/tasks/turblimp/nominalization.yaml' 2026-05-11T14:30:21,023 adding 'lm_eval/tasks/turblimp/npi_licensing.yaml' 2026-05-11T14:30:21,024 adding 'lm_eval/tasks/turblimp/passives.yaml' 2026-05-11T14:30:21,026 adding 'lm_eval/tasks/turblimp/quantifiers.yaml' 2026-05-11T14:30:21,027 adding 'lm_eval/tasks/turblimp/relative_clauses.yaml' 2026-05-11T14:30:21,028 adding 'lm_eval/tasks/turblimp/scrambling.yaml' 2026-05-11T14:30:21,029 adding 'lm_eval/tasks/turblimp/subject_agreement.yaml' 2026-05-11T14:30:21,030 adding 'lm_eval/tasks/turblimp/suspended_affixation.yaml' 2026-05-11T14:30:21,032 adding 'lm_eval/tasks/turblimp/turblimp_group.yaml' 2026-05-11T14:30:21,034 adding 'lm_eval/tasks/turkishmmlu/README.md' 2026-05-11T14:30:21,036 adding 'lm_eval/tasks/turkishmmlu/config/Biology.yaml' 2026-05-11T14:30:21,037 adding 'lm_eval/tasks/turkishmmlu/config/Chemistry.yaml' 2026-05-11T14:30:21,038 adding 'lm_eval/tasks/turkishmmlu/config/Geography.yaml' 2026-05-11T14:30:21,039 adding 'lm_eval/tasks/turkishmmlu/config/History.yaml' 2026-05-11T14:30:21,041 adding 'lm_eval/tasks/turkishmmlu/config/Mathematics.yaml' 2026-05-11T14:30:21,042 adding 'lm_eval/tasks/turkishmmlu/config/Philosophy.yaml' 2026-05-11T14:30:21,043 adding 'lm_eval/tasks/turkishmmlu/config/Physics.yaml' 2026-05-11T14:30:21,044 adding 'lm_eval/tasks/turkishmmlu/config/Religion_and_Ethics.yaml' 2026-05-11T14:30:21,045 adding 'lm_eval/tasks/turkishmmlu/config/Turkish_Language_and_Literature.yaml' 2026-05-11T14:30:21,046 adding 'lm_eval/tasks/turkishmmlu/config/_turkishmmlu_default_yaml' 2026-05-11T14:30:21,048 adding 'lm_eval/tasks/turkishmmlu/config_cot/Biology.yaml' 2026-05-11T14:30:21,050 adding 'lm_eval/tasks/turkishmmlu/config_cot/Chemistry.yaml' 2026-05-11T14:30:21,052 adding 'lm_eval/tasks/turkishmmlu/config_cot/Geography.yaml' 2026-05-11T14:30:21,054 adding 'lm_eval/tasks/turkishmmlu/config_cot/History.yaml' 2026-05-11T14:30:21,055 adding 'lm_eval/tasks/turkishmmlu/config_cot/Mathematics.yaml' 2026-05-11T14:30:21,057 adding 'lm_eval/tasks/turkishmmlu/config_cot/Philosophy.yaml' 2026-05-11T14:30:21,059 adding 'lm_eval/tasks/turkishmmlu/config_cot/Physics.yaml' 2026-05-11T14:30:21,061 adding 'lm_eval/tasks/turkishmmlu/config_cot/Religion_and_Ethics.yaml' 2026-05-11T14:30:21,062 adding 'lm_eval/tasks/turkishmmlu/config_cot/Turkish_Language_and_Literature.yaml' 2026-05-11T14:30:21,064 adding 'lm_eval/tasks/turkishmmlu/config_cot/_turkishmmlu_cot_default_yaml' 2026-05-11T14:30:21,066 adding 'lm_eval/tasks/ulqa/README.md' 2026-05-11T14:30:21,067 adding 'lm_eval/tasks/ulqa/celep1.yaml' 2026-05-11T14:30:21,068 adding 'lm_eval/tasks/ulqa/celep2.yaml' 2026-05-11T14:30:21,069 adding 'lm_eval/tasks/ulqa/lambada_uyghur.yaml' 2026-05-11T14:30:21,070 adding 'lm_eval/tasks/ulqa/uleval.yaml' 2026-05-11T14:30:21,072 adding 'lm_eval/tasks/ulqa/ulqa.yaml' 2026-05-11T14:30:21,073 adding 'lm_eval/tasks/ulqa/ulqa_.yaml' 2026-05-11T14:30:21,075 adding 'lm_eval/tasks/ulqa/ulut/nug.yaml' 2026-05-11T14:30:21,076 adding 'lm_eval/tasks/ulqa/ulut/ulut.yaml' 2026-05-11T14:30:21,077 adding 'lm_eval/tasks/ulqa/ulut/wag.yaml' 2026-05-11T14:30:21,078 adding 'lm_eval/tasks/ulqa/ulut/wsm.yaml' 2026-05-11T14:30:21,080 adding 'lm_eval/tasks/ulqa/ulut/wub.yaml' 2026-05-11T14:30:21,081 adding 'lm_eval/tasks/ulqa/ulut/wum.yaml' 2026-05-11T14:30:21,083 adding 'lm_eval/tasks/unitxt/20_newsgroups.yaml' 2026-05-11T14:30:21,084 adding 'lm_eval/tasks/unitxt/README.md' 2026-05-11T14:30:21,085 adding 'lm_eval/tasks/unitxt/ag_news.yaml' 2026-05-11T14:30:21,087 adding 'lm_eval/tasks/unitxt/argument_topic.yaml' 2026-05-11T14:30:21,088 adding 'lm_eval/tasks/unitxt/atis.yaml' 2026-05-11T14:30:21,089 adding 'lm_eval/tasks/unitxt/banking77.yaml' 2026-05-11T14:30:21,090 adding 'lm_eval/tasks/unitxt/claim_stance_topic.yaml' 2026-05-11T14:30:21,091 adding 'lm_eval/tasks/unitxt/cnn_dailymail.yaml' 2026-05-11T14:30:21,092 adding 'lm_eval/tasks/unitxt/coedit_gec.yaml' 2026-05-11T14:30:21,093 adding 'lm_eval/tasks/unitxt/dbpedia_14.yaml' 2026-05-11T14:30:21,095 adding 'lm_eval/tasks/unitxt/doc_vqa.yaml' 2026-05-11T14:30:21,096 adding 'lm_eval/tasks/unitxt/ethos_binary.yaml' 2026-05-11T14:30:21,097 adding 'lm_eval/tasks/unitxt/financial_tweets.yaml' 2026-05-11T14:30:21,098 adding 'lm_eval/tasks/unitxt/law_stack_exchange.yaml' 2026-05-11T14:30:21,099 adding 'lm_eval/tasks/unitxt/ledgar.yaml' 2026-05-11T14:30:21,100 adding 'lm_eval/tasks/unitxt/medical_abstracts.yaml' 2026-05-11T14:30:21,102 adding 'lm_eval/tasks/unitxt/stsb.yaml' 2026-05-11T14:30:21,103 adding 'lm_eval/tasks/unitxt/task.py' 2026-05-11T14:30:21,105 adding 'lm_eval/tasks/unitxt/unfair_tos.yaml' 2026-05-11T14:30:21,106 adding 'lm_eval/tasks/unitxt/unitxt' 2026-05-11T14:30:21,107 adding 'lm_eval/tasks/unitxt/unitxt_multimodal' 2026-05-11T14:30:21,108 adding 'lm_eval/tasks/unitxt/xsum.yaml' 2026-05-11T14:30:21,109 adding 'lm_eval/tasks/unitxt/yahoo_answers_topics.yaml' 2026-05-11T14:30:21,111 adding 'lm_eval/tasks/unscramble/README.md' 2026-05-11T14:30:21,113 adding 'lm_eval/tasks/unscramble/anagrams1.yaml' 2026-05-11T14:30:21,114 adding 'lm_eval/tasks/unscramble/anagrams2.yaml' 2026-05-11T14:30:21,115 adding 'lm_eval/tasks/unscramble/cycle_letters.yaml' 2026-05-11T14:30:21,116 adding 'lm_eval/tasks/unscramble/random_insertion.yaml' 2026-05-11T14:30:21,117 adding 'lm_eval/tasks/unscramble/reversed_words.yaml' 2026-05-11T14:30:21,119 adding 'lm_eval/tasks/webqs/README.md' 2026-05-11T14:30:21,120 adding 'lm_eval/tasks/webqs/utils.py' 2026-05-11T14:30:21,121 adding 'lm_eval/tasks/webqs/webqs.yaml' 2026-05-11T14:30:21,123 adding 'lm_eval/tasks/wikitext/README.md' 2026-05-11T14:30:21,124 adding 'lm_eval/tasks/wikitext/preprocess_wikitext.py' 2026-05-11T14:30:21,125 adding 'lm_eval/tasks/wikitext/wikitext.yaml' 2026-05-11T14:30:21,127 adding 'lm_eval/tasks/winogender/README.md' 2026-05-11T14:30:21,129 adding 'lm_eval/tasks/winogender/utils.py' 2026-05-11T14:30:21,130 adding 'lm_eval/tasks/winogender/winogender.yaml' 2026-05-11T14:30:21,131 adding 'lm_eval/tasks/winogender/winogender_female.yaml' 2026-05-11T14:30:21,132 adding 'lm_eval/tasks/winogender/winogender_gotcha.yaml' 2026-05-11T14:30:21,133 adding 'lm_eval/tasks/winogender/winogender_gotcha_female.yaml' 2026-05-11T14:30:21,134 adding 'lm_eval/tasks/winogender/winogender_gotcha_male.yaml' 2026-05-11T14:30:21,135 adding 'lm_eval/tasks/winogender/winogender_male.yaml' 2026-05-11T14:30:21,137 adding 'lm_eval/tasks/winogender/winogender_neutral.yaml' 2026-05-11T14:30:21,139 adding 'lm_eval/tasks/winogrande/README.md' 2026-05-11T14:30:21,140 adding 'lm_eval/tasks/winogrande/default.yaml' 2026-05-11T14:30:21,141 adding 'lm_eval/tasks/winogrande/preprocess_winogrande.py' 2026-05-11T14:30:21,143 adding 'lm_eval/tasks/wmdp/README.md' 2026-05-11T14:30:21,144 adding 'lm_eval/tasks/wmdp/_default_template_yaml' 2026-05-11T14:30:21,145 adding 'lm_eval/tasks/wmdp/_wmdp.yaml' 2026-05-11T14:30:21,146 adding 'lm_eval/tasks/wmdp/wmdp_bio.yaml' 2026-05-11T14:30:21,148 adding 'lm_eval/tasks/wmdp/wmdp_chem.yaml' 2026-05-11T14:30:21,149 adding 'lm_eval/tasks/wmdp/wmdp_cyber.yaml' 2026-05-11T14:30:21,151 adding 'lm_eval/tasks/wmt2016/README.md' 2026-05-11T14:30:21,152 adding 'lm_eval/tasks/wmt2016/metrics.py' 2026-05-11T14:30:21,153 adding 'lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml' 2026-05-11T14:30:21,155 adding 'lm_eval/tasks/wsc273/README.md' 2026-05-11T14:30:21,156 adding 'lm_eval/tasks/wsc273/default.yaml' 2026-05-11T14:30:21,157 adding 'lm_eval/tasks/wsc273/utils.py' 2026-05-11T14:30:21,159 adding 'lm_eval/tasks/xcopa/README.md' 2026-05-11T14:30:21,160 adding 'lm_eval/tasks/xcopa/_xcopa.yaml' 2026-05-11T14:30:21,162 adding 'lm_eval/tasks/xcopa/default_et.yaml' 2026-05-11T14:30:21,163 adding 'lm_eval/tasks/xcopa/default_ht.yaml' 2026-05-11T14:30:21,164 adding 'lm_eval/tasks/xcopa/default_id.yaml' 2026-05-11T14:30:21,165 adding 'lm_eval/tasks/xcopa/default_it.yaml' 2026-05-11T14:30:21,166 adding 'lm_eval/tasks/xcopa/default_qu.yaml' 2026-05-11T14:30:21,168 adding 'lm_eval/tasks/xcopa/default_sw.yaml' 2026-05-11T14:30:21,169 adding 'lm_eval/tasks/xcopa/default_ta.yaml' 2026-05-11T14:30:21,170 adding 'lm_eval/tasks/xcopa/default_th.yaml' 2026-05-11T14:30:21,171 adding 'lm_eval/tasks/xcopa/default_tr.yaml' 2026-05-11T14:30:21,172 adding 'lm_eval/tasks/xcopa/default_vi.yaml' 2026-05-11T14:30:21,173 adding 'lm_eval/tasks/xcopa/default_zh.yaml' 2026-05-11T14:30:21,175 adding 'lm_eval/tasks/xcopa/utils.py' 2026-05-11T14:30:21,177 adding 'lm_eval/tasks/xnli/README.md' 2026-05-11T14:30:21,178 adding 'lm_eval/tasks/xnli/_xnli.yaml' 2026-05-11T14:30:21,179 adding 'lm_eval/tasks/xnli/utils.py' 2026-05-11T14:30:21,181 adding 'lm_eval/tasks/xnli/xnli_ar.yaml' 2026-05-11T14:30:21,182 adding 'lm_eval/tasks/xnli/xnli_bg.yaml' 2026-05-11T14:30:21,183 adding 'lm_eval/tasks/xnli/xnli_common_yaml' 2026-05-11T14:30:21,184 adding 'lm_eval/tasks/xnli/xnli_de.yaml' 2026-05-11T14:30:21,185 adding 'lm_eval/tasks/xnli/xnli_el.yaml' 2026-05-11T14:30:21,187 adding 'lm_eval/tasks/xnli/xnli_en.yaml' 2026-05-11T14:30:21,188 adding 'lm_eval/tasks/xnli/xnli_es.yaml' 2026-05-11T14:30:21,189 adding 'lm_eval/tasks/xnli/xnli_fr.yaml' 2026-05-11T14:30:21,190 adding 'lm_eval/tasks/xnli/xnli_hi.yaml' 2026-05-11T14:30:21,192 adding 'lm_eval/tasks/xnli/xnli_ru.yaml' 2026-05-11T14:30:21,193 adding 'lm_eval/tasks/xnli/xnli_sw.yaml' 2026-05-11T14:30:21,194 adding 'lm_eval/tasks/xnli/xnli_th.yaml' 2026-05-11T14:30:21,195 adding 'lm_eval/tasks/xnli/xnli_tr.yaml' 2026-05-11T14:30:21,197 adding 'lm_eval/tasks/xnli/xnli_ur.yaml' 2026-05-11T14:30:21,198 adding 'lm_eval/tasks/xnli/xnli_vi.yaml' 2026-05-11T14:30:21,200 adding 'lm_eval/tasks/xnli/xnli_zh.yaml' 2026-05-11T14:30:21,202 adding 'lm_eval/tasks/xnli_eu/README.md' 2026-05-11T14:30:21,204 adding 'lm_eval/tasks/xnli_eu/xnli_common_yaml' 2026-05-11T14:30:21,205 adding 'lm_eval/tasks/xnli_eu/xnli_eu.yaml' 2026-05-11T14:30:21,206 adding 'lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml' 2026-05-11T14:30:21,207 adding 'lm_eval/tasks/xnli_eu/xnli_eu_native.yaml' 2026-05-11T14:30:21,209 adding 'lm_eval/tasks/xquad/README.md' 2026-05-11T14:30:21,210 adding 'lm_eval/tasks/xquad/utils.py' 2026-05-11T14:30:21,211 adding 'lm_eval/tasks/xquad/xquad_ar.yaml' 2026-05-11T14:30:21,213 adding 'lm_eval/tasks/xquad/xquad_common_yaml' 2026-05-11T14:30:21,214 adding 'lm_eval/tasks/xquad/xquad_de.yaml' 2026-05-11T14:30:21,215 adding 'lm_eval/tasks/xquad/xquad_el.yaml' 2026-05-11T14:30:21,216 adding 'lm_eval/tasks/xquad/xquad_en.yaml' 2026-05-11T14:30:21,217 adding 'lm_eval/tasks/xquad/xquad_es.yaml' 2026-05-11T14:30:21,219 adding 'lm_eval/tasks/xquad/xquad_hi.yaml' 2026-05-11T14:30:21,220 adding 'lm_eval/tasks/xquad/xquad_ro.yaml' 2026-05-11T14:30:21,221 adding 'lm_eval/tasks/xquad/xquad_ru.yaml' 2026-05-11T14:30:21,223 adding 'lm_eval/tasks/xquad/xquad_th.yaml' 2026-05-11T14:30:21,224 adding 'lm_eval/tasks/xquad/xquad_tr.yaml' 2026-05-11T14:30:21,225 adding 'lm_eval/tasks/xquad/xquad_vi.yaml' 2026-05-11T14:30:21,226 adding 'lm_eval/tasks/xquad/xquad_zh.yaml' 2026-05-11T14:30:21,228 adding 'lm_eval/tasks/xstorycloze/README.md' 2026-05-11T14:30:21,230 adding 'lm_eval/tasks/xstorycloze/_xstorycloze.yaml' 2026-05-11T14:30:21,231 adding 'lm_eval/tasks/xstorycloze/default_ar.yaml' 2026-05-11T14:30:21,232 adding 'lm_eval/tasks/xstorycloze/default_en.yaml' 2026-05-11T14:30:21,233 adding 'lm_eval/tasks/xstorycloze/default_es.yaml' 2026-05-11T14:30:21,234 adding 'lm_eval/tasks/xstorycloze/default_eu.yaml' 2026-05-11T14:30:21,235 adding 'lm_eval/tasks/xstorycloze/default_hi.yaml' 2026-05-11T14:30:21,237 adding 'lm_eval/tasks/xstorycloze/default_id.yaml' 2026-05-11T14:30:21,238 adding 'lm_eval/tasks/xstorycloze/default_my.yaml' 2026-05-11T14:30:21,239 adding 'lm_eval/tasks/xstorycloze/default_ru.yaml' 2026-05-11T14:30:21,240 adding 'lm_eval/tasks/xstorycloze/default_sw.yaml' 2026-05-11T14:30:21,241 adding 'lm_eval/tasks/xstorycloze/default_te.yaml' 2026-05-11T14:30:21,242 adding 'lm_eval/tasks/xstorycloze/default_zh.yaml' 2026-05-11T14:30:21,244 adding 'lm_eval/tasks/xwinograd/README.md' 2026-05-11T14:30:21,246 adding 'lm_eval/tasks/xwinograd/_xwinograd.yaml' 2026-05-11T14:30:21,247 adding 'lm_eval/tasks/xwinograd/utils.py' 2026-05-11T14:30:21,249 adding 'lm_eval/tasks/xwinograd/xwinograd_common_yaml' 2026-05-11T14:30:21,250 adding 'lm_eval/tasks/xwinograd/xwinograd_en.yaml' 2026-05-11T14:30:21,252 adding 'lm_eval/tasks/xwinograd/xwinograd_fr.yaml' 2026-05-11T14:30:21,253 adding 'lm_eval/tasks/xwinograd/xwinograd_jp.yaml' 2026-05-11T14:30:21,254 adding 'lm_eval/tasks/xwinograd/xwinograd_pt.yaml' 2026-05-11T14:30:21,256 adding 'lm_eval/tasks/xwinograd/xwinograd_ru.yaml' 2026-05-11T14:30:21,257 adding 'lm_eval/tasks/xwinograd/xwinograd_zh.yaml' 2026-05-11T14:30:21,261 adding 'lm_eval/tasks/zhoblimp/BA_BEI_subj_drop.yaml' 2026-05-11T14:30:21,262 adding 'lm_eval/tasks/zhoblimp/BA_deletion.yaml' 2026-05-11T14:30:21,263 adding 'lm_eval/tasks/zhoblimp/BA_duplicate_argument.yaml' 2026-05-11T14:30:21,264 adding 'lm_eval/tasks/zhoblimp/BA_inversion.yaml' 2026-05-11T14:30:21,265 adding 'lm_eval/tasks/zhoblimp/BA_meiba.yaml' 2026-05-11T14:30:21,266 adding 'lm_eval/tasks/zhoblimp/BA_negation.yaml' 2026-05-11T14:30:21,268 adding 'lm_eval/tasks/zhoblimp/BA_no_progressive.yaml' 2026-05-11T14:30:21,269 adding 'lm_eval/tasks/zhoblimp/BA_no_stative_verb.yaml' 2026-05-11T14:30:21,270 adding 'lm_eval/tasks/zhoblimp/BA_suo_adverbial_a.yaml' 2026-05-11T14:30:21,271 adding 'lm_eval/tasks/zhoblimp/BA_suo_adverbial_b.yaml' 2026-05-11T14:30:21,272 adding 'lm_eval/tasks/zhoblimp/BA_verb_le_a.yaml' 2026-05-11T14:30:21,273 adding 'lm_eval/tasks/zhoblimp/BA_verb_le_b.yaml' 2026-05-11T14:30:21,274 adding 'lm_eval/tasks/zhoblimp/BEI_construction_a.yaml' 2026-05-11T14:30:21,275 adding 'lm_eval/tasks/zhoblimp/BEI_construction_b.yaml' 2026-05-11T14:30:21,277 adding 'lm_eval/tasks/zhoblimp/BEI_deletion.yaml' 2026-05-11T14:30:21,278 adding 'lm_eval/tasks/zhoblimp/BEI_preposition.yaml' 2026-05-11T14:30:21,279 adding 'lm_eval/tasks/zhoblimp/PN_numP_a.yaml' 2026-05-11T14:30:21,280 adding 'lm_eval/tasks/zhoblimp/PN_numP_b.yaml' 2026-05-11T14:30:21,281 adding 'lm_eval/tasks/zhoblimp/README.md' 2026-05-11T14:30:21,282 adding 'lm_eval/tasks/zhoblimp/_template_yaml' 2026-05-11T14:30:21,284 adding 'lm_eval/tasks/zhoblimp/adjective_transitive_dui.yaml' 2026-05-11T14:30:21,285 adding 'lm_eval/tasks/zhoblimp/agent_animacy_adv.yaml' 2026-05-11T14:30:21,286 adding 'lm_eval/tasks/zhoblimp/agent_animacy_passive.yaml' 2026-05-11T14:30:21,287 adding 'lm_eval/tasks/zhoblimp/agent_animacy_subj.yaml' 2026-05-11T14:30:21,288 adding 'lm_eval/tasks/zhoblimp/agent_causative.yaml' 2026-05-11T14:30:21,289 adding 'lm_eval/tasks/zhoblimp/agent_deletion.yaml' 2026-05-11T14:30:21,290 adding 'lm_eval/tasks/zhoblimp/anaphor_gender_agreement.yaml' 2026-05-11T14:30:21,292 adding 'lm_eval/tasks/zhoblimp/anaphor_number_agreement.yaml' 2026-05-11T14:30:21,293 adding 'lm_eval/tasks/zhoblimp/causative_shi_ba.yaml' 2026-05-11T14:30:21,294 adding 'lm_eval/tasks/zhoblimp/classifier_noun_agreement.yaml' 2026-05-11T14:30:21,295 adding 'lm_eval/tasks/zhoblimp/classifier_noun_agreement_no_gap.yaml' 2026-05-11T14:30:21,296 adding 'lm_eval/tasks/zhoblimp/classifier_noun_subj.yaml' 2026-05-11T14:30:21,297 adding 'lm_eval/tasks/zhoblimp/control_modal_vs_raising_modal.yaml' 2026-05-11T14:30:21,298 adding 'lm_eval/tasks/zhoblimp/ellipsis_adj.yaml' 2026-05-11T14:30:21,300 adding 'lm_eval/tasks/zhoblimp/ellipsis_double_object.yaml' 2026-05-11T14:30:21,301 adding 'lm_eval/tasks/zhoblimp/ellipsis_n_bar_class.yaml' 2026-05-11T14:30:21,302 adding 'lm_eval/tasks/zhoblimp/existential_there_subject_raising.yaml' 2026-05-11T14:30:21,303 adding 'lm_eval/tasks/zhoblimp/fci_renhe_dou.yaml' 2026-05-11T14:30:21,304 adding 'lm_eval/tasks/zhoblimp/fci_renhe_prepP.yaml' 2026-05-11T14:30:21,306 adding 'lm_eval/tasks/zhoblimp/fci_renhe_ruguo.yaml' 2026-05-11T14:30:21,307 adding 'lm_eval/tasks/zhoblimp/fci_renhe_subj.yaml' 2026-05-11T14:30:21,308 adding 'lm_eval/tasks/zhoblimp/fci_renhe_suoyou.yaml' 2026-05-11T14:30:21,309 adding 'lm_eval/tasks/zhoblimp/intransitive_double_obj.yaml' 2026-05-11T14:30:21,310 adding 'lm_eval/tasks/zhoblimp/intransitive_no_obj.yaml' 2026-05-11T14:30:21,311 adding 'lm_eval/tasks/zhoblimp/left_adverbial_b.yaml' 2026-05-11T14:30:21,313 adding 'lm_eval/tasks/zhoblimp/left_adverbial_d.yaml' 2026-05-11T14:30:21,314 adding 'lm_eval/tasks/zhoblimp/left_adverbial_e.yaml' 2026-05-11T14:30:21,315 adding 'lm_eval/tasks/zhoblimp/left_adverbial_negation.yaml' 2026-05-11T14:30:21,316 adding 'lm_eval/tasks/zhoblimp/left_dou.yaml' 2026-05-11T14:30:21,317 adding 'lm_eval/tasks/zhoblimp/modal_raising_hui.yaml' 2026-05-11T14:30:21,318 adding 'lm_eval/tasks/zhoblimp/modal_raising_topicalization.yaml' 2026-05-11T14:30:21,320 adding 'lm_eval/tasks/zhoblimp/nominal_definite_men.yaml' 2026-05-11T14:30:21,321 adding 'lm_eval/tasks/zhoblimp/nominal_modal_insertion.yaml' 2026-05-11T14:30:21,322 adding 'lm_eval/tasks/zhoblimp/noun_adjective_shi.yaml' 2026-05-11T14:30:21,323 adding 'lm_eval/tasks/zhoblimp/noun_phrase_conjunction_jian.yaml' 2026-05-11T14:30:21,324 adding 'lm_eval/tasks/zhoblimp/npi_renhe_A_not_A_question.yaml' 2026-05-11T14:30:21,326 adding 'lm_eval/tasks/zhoblimp/npi_renhe_conditional.yaml' 2026-05-11T14:30:21,327 adding 'lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_locP.yaml' 2026-05-11T14:30:21,328 adding 'lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_subj.yaml' 2026-05-11T14:30:21,329 adding 'lm_eval/tasks/zhoblimp/npi_renhe_wh_question_obj.yaml' 2026-05-11T14:30:21,330 adding 'lm_eval/tasks/zhoblimp/npi_renhe_wh_question_subj.yaml' 2026-05-11T14:30:21,331 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_long_left.yaml' 2026-05-11T14:30:21,332 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_a.yaml' 2026-05-11T14:30:21,333 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_b.yaml' 2026-05-11T14:30:21,334 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_short.yaml' 2026-05-11T14:30:21,336 adding 'lm_eval/tasks/zhoblimp/passive_body_part.yaml' 2026-05-11T14:30:21,337 adding 'lm_eval/tasks/zhoblimp/passive_intransitive.yaml' 2026-05-11T14:30:21,338 adding 'lm_eval/tasks/zhoblimp/passive_no_adj.yaml' 2026-05-11T14:30:21,339 adding 'lm_eval/tasks/zhoblimp/passive_suo.yaml' 2026-05-11T14:30:21,340 adding 'lm_eval/tasks/zhoblimp/plural_cardinal_men_a.yaml' 2026-05-11T14:30:21,341 adding 'lm_eval/tasks/zhoblimp/plural_cardinal_men_b.yaml' 2026-05-11T14:30:21,343 adding 'lm_eval/tasks/zhoblimp/preposition_deletion.yaml' 2026-05-11T14:30:21,344 adding 'lm_eval/tasks/zhoblimp/preposition_insertion.yaml' 2026-05-11T14:30:21,345 adding 'lm_eval/tasks/zhoblimp/principle_A_c_command.yaml' 2026-05-11T14:30:21,346 adding 'lm_eval/tasks/zhoblimp/principle_A_c_command_number.yaml' 2026-05-11T14:30:21,347 adding 'lm_eval/tasks/zhoblimp/principle_A_domain.yaml' 2026-05-11T14:30:21,348 adding 'lm_eval/tasks/zhoblimp/principle_A_domain_number.yaml' 2026-05-11T14:30:21,349 adding 'lm_eval/tasks/zhoblimp/question_A_not_A.yaml' 2026-05-11T14:30:21,350 adding 'lm_eval/tasks/zhoblimp/question_A_not_A_daodi_a.yaml' 2026-05-11T14:30:21,352 adding 'lm_eval/tasks/zhoblimp/question_A_not_A_daodi_b.yaml' 2026-05-11T14:30:21,353 adding 'lm_eval/tasks/zhoblimp/question_A_not_A_indirect.yaml' 2026-05-11T14:30:21,354 adding 'lm_eval/tasks/zhoblimp/question_V_not_VP_1.yaml' 2026-05-11T14:30:21,355 adding 'lm_eval/tasks/zhoblimp/question_V_not_VP_2.yaml' 2026-05-11T14:30:21,356 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_1.yaml' 2026-05-11T14:30:21,357 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_2.yaml' 2026-05-11T14:30:21,358 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_intran.yaml' 2026-05-11T14:30:21,360 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_tran.yaml' 2026-05-11T14:30:21,361 adding 'lm_eval/tasks/zhoblimp/question_daodi_negation.yaml' 2026-05-11T14:30:21,362 adding 'lm_eval/tasks/zhoblimp/question_nandao_negation.yaml' 2026-05-11T14:30:21,363 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_1_a.yaml' 2026-05-11T14:30:21,364 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_1_b.yaml' 2026-05-11T14:30:21,365 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_2.yaml' 2026-05-11T14:30:21,366 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_3.yaml' 2026-05-11T14:30:21,368 adding 'lm_eval/tasks/zhoblimp/question_nandao_scope_1.yaml' 2026-05-11T14:30:21,369 adding 'lm_eval/tasks/zhoblimp/question_nandao_scope_2.yaml' 2026-05-11T14:30:21,370 adding 'lm_eval/tasks/zhoblimp/question_particle_daodi_choice_intran.yaml' 2026-05-11T14:30:21,371 adding 'lm_eval/tasks/zhoblimp/question_particle_daodi_choice_tran.yaml' 2026-05-11T14:30:21,372 adding 'lm_eval/tasks/zhoblimp/question_particle_nandao.yaml' 2026-05-11T14:30:21,374 adding 'lm_eval/tasks/zhoblimp/relative_operator_intepretation.yaml' 2026-05-11T14:30:21,375 adding 'lm_eval/tasks/zhoblimp/relative_operator_who.yaml' 2026-05-11T14:30:21,376 adding 'lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml' 2026-05-11T14:30:21,377 adding 'lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml' 2026-05-11T14:30:21,379 adding 'lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml' 2026-05-11T14:30:21,380 adding 'lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml' 2026-05-11T14:30:21,381 adding 'lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml' 2026-05-11T14:30:21,382 adding 'lm_eval/tasks/zhoblimp/right_yijing_a.yaml' 2026-05-11T14:30:21,383 adding 'lm_eval/tasks/zhoblimp/right_yijing_b.yaml' 2026-05-11T14:30:21,384 adding 'lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml' 2026-05-11T14:30:21,385 adding 'lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml' 2026-05-11T14:30:21,387 adding 'lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml' 2026-05-11T14:30:21,388 adding 'lm_eval/tasks/zhoblimp/topicalization_OSV.yaml' 2026-05-11T14:30:21,389 adding 'lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml' 2026-05-11T14:30:21,390 adding 'lm_eval/tasks/zhoblimp/topicalization_SOV.yaml' 2026-05-11T14:30:21,391 adding 'lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml' 2026-05-11T14:30:21,392 adding 'lm_eval/tasks/zhoblimp/verb_negation_particle.yaml' 2026-05-11T14:30:21,393 adding 'lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml' 2026-05-11T14:30:21,394 adding 'lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml' 2026-05-11T14:30:21,395 adding 'lm_eval/tasks/zhoblimp/ya_insertion.yaml' 2026-05-11T14:30:21,397 adding 'lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml' 2026-05-11T14:30:21,398 adding 'lm_eval/tasks/zhoblimp/you_yige.yaml' 2026-05-11T14:30:21,399 adding 'lm_eval/tasks/zhoblimp/zhoblimp_group.yaml' 2026-05-11T14:30:21,402 adding 'lm_eval-0.4.12.dist-info/licenses/LICENSE.md' 2026-05-11T14:30:21,409 adding 'lm_eval-0.4.12.dist-info/METADATA' 2026-05-11T14:30:21,410 adding 'lm_eval-0.4.12.dist-info/WHEEL' 2026-05-11T14:30:21,411 adding 'lm_eval-0.4.12.dist-info/entry_points.txt' 2026-05-11T14:30:21,412 adding 'lm_eval-0.4.12.dist-info/top_level.txt' 2026-05-11T14:30:21,673 adding 'lm_eval-0.4.12.dist-info/RECORD' 2026-05-11T14:30:22,003 removing build/bdist.linux-armv7l/wheel 2026-05-11T14:30:25,944 Building wheel for lm-eval (pyproject.toml): finished with status 'done' 2026-05-11T14:30:26,143 Created wheel for lm-eval: filename=lm_eval-0.4.12-py3-none-any.whl size=8903672 sha256=fcb805fe553443c0365cbb87826c85649f35c9f7d40fb07e0272f8629b90cb01 2026-05-11T14:30:26,144 Stored in directory: /tmp/pip-ephem-wheel-cache-gfygn71x/wheels/64/a4/d7/9c203d69f6e3b004e47819651e16b3976f4803a238842190f8 2026-05-11T14:30:26,667 Successfully built lm-eval 2026-05-11T14:30:26,881 Removed build tracker: '/tmp/pip-build-tracker-ro2flxcj'