2024-07-01T14:47:10,584 Created temporary directory: /tmp/pip-build-tracker-pqp17b_0 2024-07-01T14:47:10,585 Initialized build tracking at /tmp/pip-build-tracker-pqp17b_0 2024-07-01T14:47:10,586 Created build tracker: /tmp/pip-build-tracker-pqp17b_0 2024-07-01T14:47:10,586 Entered build tracker: /tmp/pip-build-tracker-pqp17b_0 2024-07-01T14:47:10,587 Created temporary directory: /tmp/pip-wheel-viepnuap 2024-07-01T14:47:10,590 Created temporary directory: /tmp/pip-ephem-wheel-cache-b85g5w9w 2024-07-01T14:47:10,613 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2024-07-01T14:47:10,616 2 location(s) to search for versions of lm-eval: 2024-07-01T14:47:10,616 * https://pypi.org/simple/lm-eval/ 2024-07-01T14:47:10,616 * https://www.piwheels.org/simple/lm-eval/ 2024-07-01T14:47:10,617 Fetching project page and analyzing links: https://pypi.org/simple/lm-eval/ 2024-07-01T14:47:10,618 Getting page https://pypi.org/simple/lm-eval/ 2024-07-01T14:47:10,619 Found index url https://pypi.org/simple/ 2024-07-01T14:47:10,830 Fetched page https://pypi.org/simple/lm-eval/ as application/vnd.pypi.simple.v1+json 2024-07-01T14:47:10,833 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/1b/5f/7841febb99c12ffb453d33a67b9841e89dba18c388b644bf22b81d137fc4/lm_eval-0.0.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2024-07-01T14:47:10,834 Found link https://files.pythonhosted.org/packages/21/5a/feb5ff3a1591ca963c54873d39116b0e6a4f80e493e961ac08569709c5d7/lm_eval-0.0.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.0.1 2024-07-01T14:47:10,835 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/f3/a7/63cbce8b51de25fabb1c49f3a3fd1704faaacadb5ed816401f800e4d2dbd/lm_eval-0.2.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2024-07-01T14:47:10,836 Found link https://files.pythonhosted.org/packages/c5/fd/edd21b0f258b4ec0260f99f5b2ac3864f7cddc8fb7c83bbb2379a6aab975/lm_eval-0.2.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.2.0 2024-07-01T14:47:10,836 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/61/c5/bff92e6b61fc2b0c1b7ac769633731910152e5176a404912ce7c07329ba0/lm_eval-0.3.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2024-07-01T14:47:10,837 Found link https://files.pythonhosted.org/packages/c4/f8/58abc65390a758c8c2e5f1d8bb9b58d7885d02535d5f48de27006453d07e/lm_eval-0.3.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.3.0 2024-07-01T14:47:10,838 Found link https://files.pythonhosted.org/packages/45/e0/05001c2e56e2f8f793189442176432ddb89a2f1f1ef00ea154d0bc00fe37/lm_eval-0.4.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.0 2024-07-01T14:47:10,839 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/49/2d/39f7a25ab663cb45cfc7773b85980f01df44853cc427d00dce94c90b43e6/lm_eval-0.4.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2024-07-01T14:47:10,840 Found link https://files.pythonhosted.org/packages/5a/02/1c7f1ac2f139f4c05af5b94e2c4f88a70404fa0b0c22a5fb04dec0216b03/lm_eval-0.4.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.1 2024-07-01T14:47:10,841 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/0c/5a/64cf703b62ac7ada09a514c16c7136bb4ea7ef3030eb4c3d780900a5a634/lm_eval-0.4.2-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2024-07-01T14:47:10,842 Found link https://files.pythonhosted.org/packages/75/ca/18814743ba3b42d19d8524a4f771e9c3a7aa02cd4c579747a0f513907205/lm_eval-0.4.2.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.2 2024-07-01T14:47:10,842 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/39/2b/eb15fb57ba48b3cebfd302713d680d9917101867e2b5fbfd266e88b89ff3/lm_eval-0.4.3-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2024-07-01T14:47:10,843 Found link https://files.pythonhosted.org/packages/2e/69/dbff28f17e17521735460b25645479e85b58d8ad7ea8488e4672ab52b58b/lm_eval-0.4.3.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.3 2024-07-01T14:47:10,845 Fetching project page and analyzing links: https://www.piwheels.org/simple/lm-eval/ 2024-07-01T14:47:10,845 Getting page https://www.piwheels.org/simple/lm-eval/ 2024-07-01T14:47:10,846 Found index url https://www.piwheels.org/simple/ 2024-07-01T14:47:11,002 Fetched page https://www.piwheels.org/simple/lm-eval/ as text/html 2024-07-01T14:47:11,004 Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.4.2-py3-none-any.whl#sha256=a0bcaff492a09947d8a2b80400f224113b1e313da6a688084711130da18e1af7 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2024-07-01T14:47:11,005 Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.4.1-py3-none-any.whl#sha256=796615edf52f5673187a884a9d77c7d755fe95ca2d16fe77a1f042730eb957b3 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2024-07-01T14:47:11,005 Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.4.0-py3-none-any.whl#sha256=1a863b5f478f2e66e921dbbf2e4dcda06a923021763f01b59c5b03bdd3af01c2 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2024-07-01T14:47:11,006 Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.3.0-py3-none-any.whl#sha256=498b8b8954c1f9c17f46e3ec096e9be6b9c96ee70560ee613a4eb9c7b9d31644 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2024-07-01T14:47:11,006 Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.2.0-py3-none-any.whl#sha256=e06d3d7b6016be832e6889cbc9c4787b99156ba57f8feb31d3aeb27304c6558c (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2024-07-01T14:47:11,007 Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.0.1-py3-none-any.whl#sha256=0afc289f69286f71017fb9811dfea6cda7c703bf693ad43106fbbff1f164cf14 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2024-07-01T14:47:11,008 Skipping link: not a file: https://www.piwheels.org/simple/lm-eval/ 2024-07-01T14:47:11,008 Skipping link: not a file: https://pypi.org/simple/lm-eval/ 2024-07-01T14:47:11,029 Given no hashes to check 1 links for project 'lm-eval': discarding no candidates 2024-07-01T14:47:11,048 Collecting lm-eval==0.4.3 2024-07-01T14:47:11,051 Created temporary directory: /tmp/pip-unpack-enzsg8i0 2024-07-01T14:47:11,261 Downloading lm_eval-0.4.3.tar.gz (810 kB) 2024-07-01T14:47:15,253 Added lm-eval==0.4.3 from https://files.pythonhosted.org/packages/2e/69/dbff28f17e17521735460b25645479e85b58d8ad7ea8488e4672ab52b58b/lm_eval-0.4.3.tar.gz to build tracker '/tmp/pip-build-tracker-pqp17b_0' 2024-07-01T14:47:15,260 Created temporary directory: /tmp/pip-build-env-20yvz42m 2024-07-01T14:47:15,265 Installing build dependencies: started 2024-07-01T14:47:15,266 Running command pip subprocess to install build dependencies 2024-07-01T14:47:16,490 Using pip 24.0 from /usr/local/lib/python3.11/dist-packages/pip (python 3.11) 2024-07-01T14:47:17,046 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2024-07-01T14:47:18,523 Collecting setuptools>=40.8.0 2024-07-01T14:47:18,539 Using cached https://www.piwheels.org/simple/setuptools/setuptools-70.1.1-py3-none-any.whl (883 kB) 2024-07-01T14:47:18,766 Collecting wheel 2024-07-01T14:47:18,781 Using cached https://www.piwheels.org/simple/wheel/wheel-0.43.0-py3-none-any.whl (65 kB) 2024-07-01T14:47:21,408 Installing collected packages: wheel, setuptools 2024-07-01T14:47:21,631 Creating /tmp/pip-build-env-20yvz42m/overlay/local/bin 2024-07-01T14:47:21,633 changing mode of /tmp/pip-build-env-20yvz42m/overlay/local/bin/wheel to 755 2024-07-01T14:47:24,094 Successfully installed setuptools-70.1.1 wheel-0.43.0 2024-07-01T14:47:24,448 [notice] A new release of pip is available: 24.0 -> 24.1 2024-07-01T14:47:24,449 [notice] To update, run: python3 -m pip install --upgrade pip 2024-07-01T14:47:24,720 Installing build dependencies: finished with status 'done' 2024-07-01T14:47:24,724 Getting requirements to build wheel: started 2024-07-01T14:47:24,725 Running command Getting requirements to build wheel 2024-07-01T14:47:25,713 running egg_info 2024-07-01T14:47:25,717 writing lm_eval.egg-info/PKG-INFO 2024-07-01T14:47:25,740 writing dependency_links to lm_eval.egg-info/dependency_links.txt 2024-07-01T14:47:25,741 writing entry points to lm_eval.egg-info/entry_points.txt 2024-07-01T14:47:25,753 writing requirements to lm_eval.egg-info/requires.txt 2024-07-01T14:47:25,755 writing top-level names to lm_eval.egg-info/top_level.txt 2024-07-01T14:47:26,349 reading manifest file 'lm_eval.egg-info/SOURCES.txt' 2024-07-01T14:47:26,404 adding license file 'LICENSE.md' 2024-07-01T14:47:26,530 writing manifest file 'lm_eval.egg-info/SOURCES.txt' 2024-07-01T14:47:26,657 Getting requirements to build wheel: finished with status 'done' 2024-07-01T14:47:26,660 Created temporary directory: /tmp/pip-modern-metadata-kd5ykuwb 2024-07-01T14:47:26,662 Preparing metadata (pyproject.toml): started 2024-07-01T14:47:26,664 Running command Preparing metadata (pyproject.toml) 2024-07-01T14:47:27,498 running dist_info 2024-07-01T14:47:27,502 creating /tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info 2024-07-01T14:47:27,506 writing /tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/PKG-INFO 2024-07-01T14:47:27,528 writing dependency_links to /tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/dependency_links.txt 2024-07-01T14:47:27,530 writing entry points to /tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/entry_points.txt 2024-07-01T14:47:27,542 writing requirements to /tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/requires.txt 2024-07-01T14:47:27,543 writing top-level names to /tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/top_level.txt 2024-07-01T14:47:27,545 writing manifest file '/tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/SOURCES.txt' 2024-07-01T14:47:28,084 reading manifest file '/tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/SOURCES.txt' 2024-07-01T14:47:28,087 adding license file 'LICENSE.md' 2024-07-01T14:47:28,172 writing manifest file '/tmp/pip-modern-metadata-kd5ykuwb/lm_eval.egg-info/SOURCES.txt' 2024-07-01T14:47:28,177 creating '/tmp/pip-modern-metadata-kd5ykuwb/lm_eval-0.4.3.dist-info' 2024-07-01T14:47:28,354 Preparing metadata (pyproject.toml): finished with status 'done' 2024-07-01T14:47:28,362 Source in /tmp/pip-wheel-viepnuap/lm-eval_8dffd256ff4543a0a8cff6a1447caf97 has version 0.4.3, which satisfies requirement lm-eval==0.4.3 from https://files.pythonhosted.org/packages/2e/69/dbff28f17e17521735460b25645479e85b58d8ad7ea8488e4672ab52b58b/lm_eval-0.4.3.tar.gz 2024-07-01T14:47:28,363 Removed lm-eval==0.4.3 from https://files.pythonhosted.org/packages/2e/69/dbff28f17e17521735460b25645479e85b58d8ad7ea8488e4672ab52b58b/lm_eval-0.4.3.tar.gz from build tracker '/tmp/pip-build-tracker-pqp17b_0' 2024-07-01T14:47:28,373 Created temporary directory: /tmp/pip-unpack-3ze79aeh 2024-07-01T14:47:28,374 Created temporary directory: /tmp/pip-unpack-kbf2z1ck 2024-07-01T14:47:28,543 Building wheels for collected packages: lm-eval 2024-07-01T14:47:28,547 Created temporary directory: /tmp/pip-wheel-3mimzwor 2024-07-01T14:47:28,548 Destination directory: /tmp/pip-wheel-3mimzwor 2024-07-01T14:47:28,550 Building wheel for lm-eval (pyproject.toml): started 2024-07-01T14:47:28,551 Running command Building wheel for lm-eval (pyproject.toml) 2024-07-01T14:47:29,430 running bdist_wheel 2024-07-01T14:47:29,446 running build 2024-07-01T14:47:29,446 running build_py 2024-07-01T14:47:29,450 creating build 2024-07-01T14:47:29,451 creating build/lib 2024-07-01T14:47:29,452 creating build/lib/lm_eval 2024-07-01T14:47:29,453 copying lm_eval/evaluator.py -> build/lib/lm_eval 2024-07-01T14:47:29,455 copying lm_eval/evaluator_utils.py -> build/lib/lm_eval 2024-07-01T14:47:29,458 copying lm_eval/utils.py -> build/lib/lm_eval 2024-07-01T14:47:29,460 copying lm_eval/__init__.py -> build/lib/lm_eval 2024-07-01T14:47:29,462 copying lm_eval/__main__.py -> build/lib/lm_eval 2024-07-01T14:47:29,464 creating build/lib/lm_eval/decontamination 2024-07-01T14:47:29,465 copying lm_eval/decontamination/archiver.py -> build/lib/lm_eval/decontamination 2024-07-01T14:47:29,467 copying lm_eval/decontamination/janitor.py -> build/lib/lm_eval/decontamination 2024-07-01T14:47:29,470 copying lm_eval/decontamination/__init__.py -> build/lib/lm_eval/decontamination 2024-07-01T14:47:29,471 copying lm_eval/decontamination/decontaminate.py -> build/lib/lm_eval/decontamination 2024-07-01T14:47:29,474 creating build/lib/lm_eval/prompts 2024-07-01T14:47:29,474 copying lm_eval/prompts/__init__.py -> build/lib/lm_eval/prompts 2024-07-01T14:47:29,477 creating build/lib/lm_eval/tasks 2024-07-01T14:47:29,478 copying lm_eval/tasks/__init__.py -> build/lib/lm_eval/tasks 2024-07-01T14:47:29,481 creating build/lib/lm_eval/api 2024-07-01T14:47:29,482 copying lm_eval/api/instance.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,484 copying lm_eval/api/samplers.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,486 copying lm_eval/api/task.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,490 copying lm_eval/api/registry.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,492 copying lm_eval/api/__init__.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,493 copying lm_eval/api/filter.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,495 copying lm_eval/api/model.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,498 copying lm_eval/api/metrics.py -> build/lib/lm_eval/api 2024-07-01T14:47:29,501 creating build/lib/lm_eval/caching 2024-07-01T14:47:29,502 copying lm_eval/caching/cache.py -> build/lib/lm_eval/caching 2024-07-01T14:47:29,505 creating build/lib/lm_eval/models 2024-07-01T14:47:29,506 copying lm_eval/models/gguf.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,508 copying lm_eval/models/neuralmagic.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,510 copying lm_eval/models/anthropic_llms.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,513 copying lm_eval/models/utils.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,515 copying lm_eval/models/vllm_causallms.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,518 copying lm_eval/models/dummy.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,520 copying lm_eval/models/textsynth.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,522 copying lm_eval/models/__init__.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,523 copying lm_eval/models/nemo_lm.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,526 copying lm_eval/models/optimum_lm.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,528 copying lm_eval/models/openai_completions.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,530 copying lm_eval/models/neuron_optimum.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,533 copying lm_eval/models/mamba_lm.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,535 copying lm_eval/models/huggingface.py -> build/lib/lm_eval/models 2024-07-01T14:47:29,538 creating build/lib/lm_eval/filters 2024-07-01T14:47:29,539 copying lm_eval/filters/extraction.py -> build/lib/lm_eval/filters 2024-07-01T14:47:29,541 copying lm_eval/filters/transformation.py -> build/lib/lm_eval/filters 2024-07-01T14:47:29,543 copying lm_eval/filters/decontamination.py -> build/lib/lm_eval/filters 2024-07-01T14:47:29,545 copying lm_eval/filters/__init__.py -> build/lib/lm_eval/filters 2024-07-01T14:47:29,548 copying lm_eval/filters/selection.py -> build/lib/lm_eval/filters 2024-07-01T14:47:29,550 creating build/lib/lm_eval/loggers 2024-07-01T14:47:29,551 copying lm_eval/loggers/utils.py -> build/lib/lm_eval/loggers 2024-07-01T14:47:29,553 copying lm_eval/loggers/__init__.py -> build/lib/lm_eval/loggers 2024-07-01T14:47:29,555 copying lm_eval/loggers/wandb_logger.py -> build/lib/lm_eval/loggers 2024-07-01T14:47:29,557 copying lm_eval/loggers/evaluation_tracker.py -> build/lib/lm_eval/loggers 2024-07-01T14:47:29,560 creating build/lib/lm_eval/tasks/webqs 2024-07-01T14:47:29,561 copying lm_eval/tasks/webqs/utils.py -> build/lib/lm_eval/tasks/webqs 2024-07-01T14:47:29,564 creating build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:29,565 copying lm_eval/tasks/unitxt/unitxt_wrapper.py -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:29,566 copying lm_eval/tasks/unitxt/generate_yamls.py -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:29,569 creating build/lib/lm_eval/tasks/medmcqa 2024-07-01T14:47:29,570 copying lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/lib/lm_eval/tasks/medmcqa 2024-07-01T14:47:29,572 creating build/lib/lm_eval/tasks/realtoxicityprompts 2024-07-01T14:47:29,573 copying lm_eval/tasks/realtoxicityprompts/metric.py -> build/lib/lm_eval/tasks/realtoxicityprompts 2024-07-01T14:47:29,575 creating build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:29,576 copying lm_eval/tasks/eus_exams/configs.py -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:29,578 copying lm_eval/tasks/eus_exams/utils.py -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:29,580 creating build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:29,581 copying lm_eval/tasks/cmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:29,584 creating build/lib/lm_eval/tasks/squad_completion 2024-07-01T14:47:29,585 copying lm_eval/tasks/squad_completion/task.py -> build/lib/lm_eval/tasks/squad_completion 2024-07-01T14:47:29,587 creating build/lib/lm_eval/tasks/wsc273 2024-07-01T14:47:29,588 copying lm_eval/tasks/wsc273/utils.py -> build/lib/lm_eval/tasks/wsc273 2024-07-01T14:47:29,592 creating build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:29,593 copying lm_eval/tasks/csatqa/utils.py -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:29,595 copying lm_eval/tasks/csatqa/_generate_configs.py -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:29,597 creating build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:29,598 copying lm_eval/tasks/basqueglue/utils.py -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:29,600 creating build/lib/lm_eval/tasks/eq_bench 2024-07-01T14:47:29,601 copying lm_eval/tasks/eq_bench/utils.py -> build/lib/lm_eval/tasks/eq_bench 2024-07-01T14:47:29,603 creating build/lib/lm_eval/tasks/mmlu 2024-07-01T14:47:29,604 copying lm_eval/tasks/mmlu/_generate_configs.py -> build/lib/lm_eval/tasks/mmlu 2024-07-01T14:47:29,607 creating build/lib/lm_eval/tasks/wikitext 2024-07-01T14:47:29,608 copying lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/lib/lm_eval/tasks/wikitext 2024-07-01T14:47:29,610 creating build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:29,611 copying lm_eval/tasks/hendrycks_ethics/utils.py -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:29,614 creating build/lib/lm_eval/tasks/eus_reading 2024-07-01T14:47:29,615 copying lm_eval/tasks/eus_reading/utils.py -> build/lib/lm_eval/tasks/eus_reading 2024-07-01T14:47:29,619 creating build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:29,620 copying lm_eval/tasks/xwinograd/utils.py -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:29,623 creating build/lib/lm_eval/tasks/hellaswag 2024-07-01T14:47:29,624 copying lm_eval/tasks/hellaswag/utils.py -> build/lib/lm_eval/tasks/hellaswag 2024-07-01T14:47:29,627 creating build/lib/lm_eval/tasks/fda 2024-07-01T14:47:29,628 copying lm_eval/tasks/fda/task.py -> build/lib/lm_eval/tasks/fda 2024-07-01T14:47:29,630 creating build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:29,631 copying lm_eval/tasks/xcopa/utils.py -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:29,634 creating build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:29,635 copying lm_eval/tasks/hendrycks_math/utils.py -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:29,638 creating build/lib/lm_eval/tasks/toxigen 2024-07-01T14:47:29,639 copying lm_eval/tasks/toxigen/utils.py -> build/lib/lm_eval/tasks/toxigen 2024-07-01T14:47:29,641 creating build/lib/lm_eval/tasks/eus_trivia 2024-07-01T14:47:29,642 copying lm_eval/tasks/eus_trivia/utils.py -> build/lib/lm_eval/tasks/eus_trivia 2024-07-01T14:47:29,645 creating build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:29,646 copying lm_eval/tasks/ceval/_generate_configs.py -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:29,648 creating build/lib/lm_eval/tasks/coqa 2024-07-01T14:47:29,649 copying lm_eval/tasks/coqa/utils.py -> build/lib/lm_eval/tasks/coqa 2024-07-01T14:47:29,652 creating build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:29,652 copying lm_eval/tasks/french_bench/utils.py -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:29,654 copying lm_eval/tasks/french_bench/preprocess_wikitext.py -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:29,656 creating build/lib/lm_eval/tasks/squadv2 2024-07-01T14:47:29,657 copying lm_eval/tasks/squadv2/task.py -> build/lib/lm_eval/tasks/squadv2 2024-07-01T14:47:29,660 creating build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:29,661 copying lm_eval/tasks/belebele/_generate_configs.py -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:29,663 creating build/lib/lm_eval/tasks/bigbench 2024-07-01T14:47:29,664 copying lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/lib/lm_eval/tasks/bigbench 2024-07-01T14:47:29,666 copying lm_eval/tasks/bigbench/generate_tasks.py -> build/lib/lm_eval/tasks/bigbench 2024-07-01T14:47:29,669 creating build/lib/lm_eval/tasks/logiqa 2024-07-01T14:47:29,670 copying lm_eval/tasks/logiqa/utils_logiqa.py -> build/lib/lm_eval/tasks/logiqa 2024-07-01T14:47:29,672 creating build/lib/lm_eval/tasks/glianorex 2024-07-01T14:47:29,673 copying lm_eval/tasks/glianorex/preprocess_glianorex.py -> build/lib/lm_eval/tasks/glianorex 2024-07-01T14:47:29,675 creating build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:29,676 copying lm_eval/tasks/crows_pairs/utils.py -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:29,679 creating build/lib/lm_eval/tasks/qasper 2024-07-01T14:47:29,679 copying lm_eval/tasks/qasper/utils.py -> build/lib/lm_eval/tasks/qasper 2024-07-01T14:47:29,681 copying lm_eval/tasks/qasper/metrics.py -> build/lib/lm_eval/tasks/qasper 2024-07-01T14:47:29,683 creating build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:29,684 copying lm_eval/tasks/scrolls/task.py -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:29,687 creating build/lib/lm_eval/tasks/mathqa 2024-07-01T14:47:29,688 copying lm_eval/tasks/mathqa/utils.py -> build/lib/lm_eval/tasks/mathqa 2024-07-01T14:47:29,690 creating build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:29,691 copying lm_eval/tasks/paloma/paloma_utils.py -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:29,694 creating build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:29,695 copying lm_eval/tasks/agieval/utils.py -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:29,698 creating build/lib/lm_eval/tasks/winogrande 2024-07-01T14:47:29,699 copying lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/lib/lm_eval/tasks/winogrande 2024-07-01T14:47:29,701 creating build/lib/lm_eval/tasks/race 2024-07-01T14:47:29,702 copying lm_eval/tasks/race/preprocess_race.py -> build/lib/lm_eval/tasks/race 2024-07-01T14:47:29,705 creating build/lib/lm_eval/tasks/bbh 2024-07-01T14:47:29,705 copying lm_eval/tasks/bbh/_generate_configs.py -> build/lib/lm_eval/tasks/bbh 2024-07-01T14:47:29,708 creating build/lib/lm_eval/tasks/mgsm 2024-07-01T14:47:29,709 copying lm_eval/tasks/mgsm/utils.py -> build/lib/lm_eval/tasks/mgsm 2024-07-01T14:47:29,712 creating build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:29,713 copying lm_eval/tasks/aclue/_generate_configs.py -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:29,717 creating build/lib/lm_eval/tasks/truthfulqa 2024-07-01T14:47:29,718 copying lm_eval/tasks/truthfulqa/utils.py -> build/lib/lm_eval/tasks/truthfulqa 2024-07-01T14:47:29,721 creating build/lib/lm_eval/tasks/translation 2024-07-01T14:47:29,722 copying lm_eval/tasks/translation/utils.py -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:29,724 creating build/lib/lm_eval/tasks/qa4mre 2024-07-01T14:47:29,725 copying lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/lib/lm_eval/tasks/qa4mre 2024-07-01T14:47:29,727 creating build/lib/lm_eval/tasks/copal_id 2024-07-01T14:47:29,728 copying lm_eval/tasks/copal_id/utils.py -> build/lib/lm_eval/tasks/copal_id 2024-07-01T14:47:29,731 creating build/lib/lm_eval/tasks/wmt2016 2024-07-01T14:47:29,732 copying lm_eval/tasks/wmt2016/metrics.py -> build/lib/lm_eval/tasks/wmt2016 2024-07-01T14:47:29,734 creating build/lib/lm_eval/tasks/drop 2024-07-01T14:47:29,735 copying lm_eval/tasks/drop/utils.py -> build/lib/lm_eval/tasks/drop 2024-07-01T14:47:29,738 creating build/lib/lm_eval/tasks/mutual 2024-07-01T14:47:29,739 copying lm_eval/tasks/mutual/utils.py -> build/lib/lm_eval/tasks/mutual 2024-07-01T14:47:29,742 creating build/lib/lm_eval/tasks/swde 2024-07-01T14:47:29,743 copying lm_eval/tasks/swde/task.py -> build/lib/lm_eval/tasks/swde 2024-07-01T14:47:29,745 creating build/lib/lm_eval/tasks/noticia 2024-07-01T14:47:29,746 copying lm_eval/tasks/noticia/utils.py -> build/lib/lm_eval/tasks/noticia 2024-07-01T14:47:29,751 creating build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:29,752 copying lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:29,754 copying lm_eval/tasks/tinyBenchmarks/agg_functions.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:29,755 copying lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:29,757 copying lm_eval/tasks/tinyBenchmarks/utils_winogrande.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:29,760 creating build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:29,760 copying lm_eval/tasks/kobest/utils.py -> build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:29,763 creating build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:29,764 copying lm_eval/tasks/xnli/utils.py -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:29,767 creating build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:29,768 copying lm_eval/tasks/minerva_math/utils.py -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:29,771 creating build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:29,772 copying lm_eval/tasks/paws-x/_generate_config.py -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:29,774 creating build/lib/lm_eval/tasks/pubmedqa 2024-07-01T14:47:29,775 copying lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/lib/lm_eval/tasks/pubmedqa 2024-07-01T14:47:29,778 creating build/lib/lm_eval/tasks/ifeval 2024-07-01T14:47:29,779 copying lm_eval/tasks/ifeval/utils.py -> build/lib/lm_eval/tasks/ifeval 2024-07-01T14:47:29,781 copying lm_eval/tasks/ifeval/instructions.py -> build/lib/lm_eval/tasks/ifeval 2024-07-01T14:47:29,785 copying lm_eval/tasks/ifeval/instructions_util.py -> build/lib/lm_eval/tasks/ifeval 2024-07-01T14:47:29,788 copying lm_eval/tasks/ifeval/instructions_registry.py -> build/lib/lm_eval/tasks/ifeval 2024-07-01T14:47:29,795 creating build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:29,796 copying lm_eval/tasks/arabicmmlu/utils.py -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:29,798 copying lm_eval/tasks/arabicmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:29,800 creating build/lib/lm_eval/tasks/logiqa2 2024-07-01T14:47:29,801 copying lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/lib/lm_eval/tasks/logiqa2 2024-07-01T14:47:29,804 creating build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:29,805 copying lm_eval/tasks/blimp/generate_configs.py -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:29,808 creating build/lib/lm_eval/tasks/medqa 2024-07-01T14:47:29,809 copying lm_eval/tasks/medqa/preprocess_medqa.py -> build/lib/lm_eval/tasks/medqa 2024-07-01T14:47:29,811 creating build/lib/lm_eval/tasks/gpqa 2024-07-01T14:47:29,812 creating build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:29,813 copying lm_eval/tasks/gpqa/cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:29,814 copying lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:29,816 creating build/lib/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:29,817 copying lm_eval/tasks/gpqa/n_shot/utils.py -> build/lib/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:29,820 copying lm_eval/tasks/gpqa/n_shot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:29,823 creating build/lib/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:29,824 copying lm_eval/tasks/gpqa/zeroshot/utils.py -> build/lib/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:29,826 copying lm_eval/tasks/gpqa/zeroshot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:29,828 creating build/lib/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:29,828 copying lm_eval/tasks/gpqa/generative/utils.py -> build/lib/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:29,830 copying lm_eval/tasks/gpqa/generative/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:29,833 creating build/lib/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:29,834 copying lm_eval/tasks/gpqa/cot_n_shot/utils.py -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:29,835 copying lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:29,838 creating build/lib/lm_eval/tasks/tmmluplus 2024-07-01T14:47:29,839 creating build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:29,840 copying lm_eval/tasks/tmmluplus/default/utils.py -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:29,842 copying lm_eval/tasks/tmmluplus/default/_generate_configs.py -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:29,845 creating build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:29,846 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:29,851 creating build/lib/lm_eval/tasks/mmlu/flan_n_shot 2024-07-01T14:47:29,852 creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:29,853 copying lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:29,858 creating build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:29,859 copying lm_eval/tasks/bbh/cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:29,862 creating build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:29,863 copying lm_eval/tasks/bbh/zeroshot/utils.py -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:29,869 creating build/lib/lm_eval/tasks/glue 2024-07-01T14:47:29,869 creating build/lib/lm_eval/tasks/glue/mnli 2024-07-01T14:47:29,870 copying lm_eval/tasks/glue/mnli/utils.py -> build/lib/lm_eval/tasks/glue/mnli 2024-07-01T14:47:29,874 creating build/lib/lm_eval/tasks/okapi 2024-07-01T14:47:29,874 creating build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:29,875 copying lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:29,878 creating build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:29,879 copying lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:29,881 creating build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:29,882 copying lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:29,884 creating build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:29,885 copying lm_eval/tasks/okapi/arc_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:29,887 creating build/lib/lm_eval/tasks/code_x_glue 2024-07-01T14:47:29,888 creating build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:29,889 copying lm_eval/tasks/code_x_glue/code-text/utils.py -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:29,890 copying lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:29,893 creating build/lib/lm_eval/tasks/model_written_evals 2024-07-01T14:47:29,894 creating build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:29,895 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:29,898 creating build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:29,899 copying lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:29,903 creating build/lib/lm_eval/tasks/super_glue 2024-07-01T14:47:29,904 creating build/lib/lm_eval/tasks/super_glue/record 2024-07-01T14:47:29,905 copying lm_eval/tasks/super_glue/record/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/record 2024-07-01T14:47:29,907 copying lm_eval/tasks/super_glue/record/util.py -> build/lib/lm_eval/tasks/super_glue/record 2024-07-01T14:47:29,910 creating build/lib/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:29,910 copying lm_eval/tasks/super_glue/copa/utils.py -> build/lib/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:29,913 creating build/lib/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:29,914 copying lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:29,916 copying lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/lib/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:29,918 creating build/lib/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:29,919 copying lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:29,921 creating build/lib/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:29,922 copying lm_eval/tasks/super_glue/cb/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:29,923 copying lm_eval/tasks/super_glue/cb/aggregate.py -> build/lib/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:29,928 running egg_info 2024-07-01T14:47:29,931 writing lm_eval.egg-info/PKG-INFO 2024-07-01T14:47:29,953 writing dependency_links to lm_eval.egg-info/dependency_links.txt 2024-07-01T14:47:29,955 writing entry points to lm_eval.egg-info/entry_points.txt 2024-07-01T14:47:29,966 writing requirements to lm_eval.egg-info/requires.txt 2024-07-01T14:47:29,967 writing top-level names to lm_eval.egg-info/top_level.txt 2024-07-01T14:47:30,493 reading manifest file 'lm_eval.egg-info/SOURCES.txt' 2024-07-01T14:47:30,549 adding license file 'LICENSE.md' 2024-07-01T14:47:30,671 writing manifest file 'lm_eval.egg-info/SOURCES.txt' 2024-07-01T14:47:31,217 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:31,219 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:31,221 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:31,223 copying lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:31,225 copying lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:31,227 copying lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:31,229 copying lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:31,231 copying lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:31,233 copying lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:31,235 copying lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:31,237 copying lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:31,238 copying lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:31,240 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:31,242 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:31,244 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:31,246 copying lm_eval/tasks/webqs/webqs.yaml -> build/lib/lm_eval/tasks/webqs 2024-07-01T14:47:31,248 copying lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,250 copying lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,252 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,254 copying lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,256 copying lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,258 copying lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,259 copying lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,262 copying lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,264 copying lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,266 copying lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,267 copying lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,270 copying lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,271 copying lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,273 copying lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,275 copying lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,277 copying lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,279 copying lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,282 copying lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,284 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,286 copying lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,288 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,290 copying lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,292 copying lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,294 copying lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,296 copying lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,298 copying lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,300 copying lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,302 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,304 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,306 copying lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,308 copying lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,310 copying lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,312 copying lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,314 copying lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,316 copying lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,318 copying lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,320 copying lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,322 copying lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,324 copying lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,326 copying lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,328 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,330 copying lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,332 copying lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,335 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,337 copying lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,339 copying lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,341 copying lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,343 copying lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,345 copying lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,347 copying lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,349 copying lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,351 copying lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,353 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,355 copying lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,357 copying lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,359 copying lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,361 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,364 copying lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,366 copying lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,368 copying lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,370 copying lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,372 copying lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,374 copying lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,376 copying lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,378 copying lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,380 copying lm_eval/tasks/tmmluplus/default/tmmluplus.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,382 copying lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,384 copying lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:31,386 copying lm_eval/tasks/unitxt/20_newsgroups.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,388 copying lm_eval/tasks/unitxt/cnn_dailymail.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,390 copying lm_eval/tasks/unitxt/dbpedia_14.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,392 copying lm_eval/tasks/unitxt/ag_news.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,394 copying lm_eval/tasks/unitxt/medical_abstracts.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,396 copying lm_eval/tasks/unitxt/yahoo_answers_topics.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,398 copying lm_eval/tasks/unitxt/unfair_tos.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,400 copying lm_eval/tasks/unitxt/financial_tweets.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,402 copying lm_eval/tasks/unitxt/stsb.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,404 copying lm_eval/tasks/unitxt/ethos_binary.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,406 copying lm_eval/tasks/unitxt/xsum.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,408 copying lm_eval/tasks/unitxt/banking77.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,410 copying lm_eval/tasks/unitxt/law_stack_exchange.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,413 copying lm_eval/tasks/unitxt/atis.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,415 copying lm_eval/tasks/unitxt/coedit_gec.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,417 copying lm_eval/tasks/unitxt/claim_stance_topic.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,419 copying lm_eval/tasks/unitxt/argument_topic.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,421 copying lm_eval/tasks/unitxt/ledgar.yaml -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:31,422 creating build/lib/lm_eval/tasks/sciq 2024-07-01T14:47:31,423 copying lm_eval/tasks/sciq/sciq.yaml -> build/lib/lm_eval/tasks/sciq 2024-07-01T14:47:31,426 copying lm_eval/tasks/medmcqa/medmcqa.yaml -> build/lib/lm_eval/tasks/medmcqa 2024-07-01T14:47:31,428 copying lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/lib/lm_eval/tasks/realtoxicityprompts 2024-07-01T14:47:31,429 creating build/lib/lm_eval/tasks/headqa 2024-07-01T14:47:31,430 copying lm_eval/tasks/headqa/headqa_es.yaml -> build/lib/lm_eval/tasks/headqa 2024-07-01T14:47:31,432 copying lm_eval/tasks/headqa/headqa_en.yaml -> build/lib/lm_eval/tasks/headqa 2024-07-01T14:47:31,435 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,437 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,439 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,441 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,443 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,445 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,447 copying lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,449 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,451 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,453 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,455 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,457 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,459 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,462 copying lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,464 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,466 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,468 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,470 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,472 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,474 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,477 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,479 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,481 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,483 copying lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,485 copying lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,487 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,490 copying lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,491 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,494 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,495 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,498 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,499 copying lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,502 copying lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,504 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,506 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,508 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,510 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,512 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,514 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,516 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,518 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,520 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,522 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,525 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,527 copying lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,529 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,531 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,533 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,535 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,538 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,540 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,542 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,544 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,546 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,548 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,551 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,553 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,555 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,557 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,558 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,560 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,562 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:31,565 copying lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,566 copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_commonsense.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,568 copying lm_eval/tasks/cmmlu/cmmlu_default_anatomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,570 copying lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,572 copying lm_eval/tasks/cmmlu/cmmlu_default_college_medical_statistics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,574 copying lm_eval/tasks/cmmlu/cmmlu_default_logical.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,576 copying lm_eval/tasks/cmmlu/cmmlu_default_food_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,579 copying lm_eval/tasks/cmmlu/cmmlu_default_philosophy.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,581 copying lm_eval/tasks/cmmlu/cmmlu_default_computer_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,583 copying lm_eval/tasks/cmmlu/cmmlu_default_college_education.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,585 copying lm_eval/tasks/cmmlu/cmmlu_default_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,587 copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_foreign_policy.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,589 copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,591 copying lm_eval/tasks/cmmlu/cmmlu_default_professional_accounting.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,594 copying lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,595 copying lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,598 copying lm_eval/tasks/cmmlu/cmmlu_default_education.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,600 copying lm_eval/tasks/cmmlu/cmmlu_default_nutrition.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,602 copying lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,604 copying lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,606 copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,608 copying lm_eval/tasks/cmmlu/cmmlu_default_professional_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,610 copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_politics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,612 copying lm_eval/tasks/cmmlu/cmmlu_default_college_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,614 copying lm_eval/tasks/cmmlu/cmmlu_default_ancient_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,616 copying lm_eval/tasks/cmmlu/cmmlu_default_college_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,618 copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_physics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,620 copying lm_eval/tasks/cmmlu/cmmlu_default_legal_and_moral_basis.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,622 copying lm_eval/tasks/cmmlu/cmmlu_default_jurisprudence.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,624 copying lm_eval/tasks/cmmlu/cmmlu_default_genetics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,626 copying lm_eval/tasks/cmmlu/cmmlu_default_international_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,628 copying lm_eval/tasks/cmmlu/cmmlu_default_ethnology.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,630 copying lm_eval/tasks/cmmlu/cmmlu_default_modern_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,632 copying lm_eval/tasks/cmmlu/cmmlu_default_electrical_engineering.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,634 copying lm_eval/tasks/cmmlu/cmmlu_default_management.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,636 copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_civil_service_exam.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,638 copying lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,640 copying lm_eval/tasks/cmmlu/cmmlu_default_journalism.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,642 copying lm_eval/tasks/cmmlu/cmmlu_default_construction_project_management.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,644 copying lm_eval/tasks/cmmlu/cmmlu_default_machine_learning.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,647 copying lm_eval/tasks/cmmlu/cmmlu_default_college_engineering_hydrology.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,649 copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_literature.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,651 copying lm_eval/tasks/cmmlu/cmmlu_default_business_ethics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,653 copying lm_eval/tasks/cmmlu/cmmlu_default_professional_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,655 copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_geography.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,657 copying lm_eval/tasks/cmmlu/cmmlu_default_college_actuarial_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,659 copying lm_eval/tasks/cmmlu/cmmlu_default_college_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,661 copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_information_and_technology.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,663 copying lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,666 copying lm_eval/tasks/cmmlu/cmmlu_default_human_sexuality.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,668 copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_teacher_qualification.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,670 copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_history.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,672 copying lm_eval/tasks/cmmlu/cmmlu_default_agronomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,674 copying lm_eval/tasks/cmmlu/cmmlu_default_marketing.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,676 copying lm_eval/tasks/cmmlu/cmmlu_default_arts.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,678 copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_driving_rule.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,680 copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,683 copying lm_eval/tasks/cmmlu/cmmlu_default_computer_security.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,685 copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_food_culture.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,687 copying lm_eval/tasks/cmmlu/cmmlu_default_global_facts.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,689 copying lm_eval/tasks/cmmlu/cmmlu_default_astronomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,691 copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,693 copying lm_eval/tasks/cmmlu/cmmlu_default_marxist_theory.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,695 copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_biology.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,697 copying lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,699 copying lm_eval/tasks/cmmlu/cmmlu_default_conceptual_physics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,700 copying lm_eval/tasks/cmmlu/cmmlu_default_economics.yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:31,703 creating build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:31,704 copying lm_eval/tasks/haerae/haerae_lw.yaml -> build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:31,706 copying lm_eval/tasks/haerae/haerae_sn.yaml -> build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:31,708 copying lm_eval/tasks/haerae/haerae_hi.yaml -> build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:31,710 copying lm_eval/tasks/haerae/haerae_rw.yaml -> build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:31,712 copying lm_eval/tasks/haerae/haerae_gk.yaml -> build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:31,714 copying lm_eval/tasks/squad_completion/squad_completion.yaml -> build/lib/lm_eval/tasks/squad_completion 2024-07-01T14:47:31,716 copying lm_eval/tasks/wsc273/default.yaml -> build/lib/lm_eval/tasks/wsc273 2024-07-01T14:47:31,718 copying lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:31,720 copying lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:31,722 copying lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:31,724 copying lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:31,726 copying lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:31,728 copying lm_eval/tasks/csatqa/csatqa_li.yaml -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:31,730 copying lm_eval/tasks/basqueglue/vaxx.yaml -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:31,732 copying lm_eval/tasks/basqueglue/qnli.yaml -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:31,734 copying lm_eval/tasks/basqueglue/wic.yaml -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:31,736 copying lm_eval/tasks/basqueglue/coref.yaml -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:31,738 copying lm_eval/tasks/basqueglue/bhtc.yaml -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:31,740 copying lm_eval/tasks/basqueglue/bec.yaml -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:31,742 copying lm_eval/tasks/eq_bench/default.yaml -> build/lib/lm_eval/tasks/eq_bench 2024-07-01T14:47:31,744 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,746 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,748 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,750 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,752 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,755 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,757 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,759 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,761 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,763 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,765 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,767 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,769 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,771 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,774 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,775 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,778 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,780 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,782 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,784 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,786 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,788 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,790 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,792 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,794 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,796 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,798 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,800 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,802 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,804 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,806 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,809 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,811 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,813 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,815 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,817 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,819 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,821 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,823 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,825 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,827 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,829 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,831 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,833 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,835 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,837 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,839 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,841 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,844 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,846 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,848 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,850 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,852 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,854 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,857 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,859 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,861 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,863 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:31,865 creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,866 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,868 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,870 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,873 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,875 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,877 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,879 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,881 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,883 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,885 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,887 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,889 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,891 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,893 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,896 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,898 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,900 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,902 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,905 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,907 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,910 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,912 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,914 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,917 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,919 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,922 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,925 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,927 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,929 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,931 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,933 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,935 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,937 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,939 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,941 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,942 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,944 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,947 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,949 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,951 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,953 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,955 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,957 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,959 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,961 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,963 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,965 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,967 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,970 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,972 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,974 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,976 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,979 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,981 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,983 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,984 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,987 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,988 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:31,990 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:31,992 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:31,994 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:31,996 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:31,998 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,000 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,002 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,004 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,006 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,009 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,011 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,013 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,015 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,017 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,019 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,021 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,023 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,026 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,028 copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,030 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,032 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,034 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,036 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,038 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,040 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,042 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,043 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,046 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,048 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,050 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,052 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,054 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,056 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,059 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,061 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,064 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,066 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,069 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,115 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,117 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,119 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,121 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,123 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,125 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,127 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,129 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,131 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,133 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,135 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,137 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,139 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,141 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,143 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,145 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,147 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,149 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,151 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,153 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:32,155 creating build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,156 copying lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,158 copying lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,160 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,163 copying lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,165 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,167 copying lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,169 copying lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,171 copying lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,173 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,175 copying lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,177 copying lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,179 copying lm_eval/tasks/mmlu/continuation/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,182 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,184 copying lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,186 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,188 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,190 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,192 copying lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,194 copying lm_eval/tasks/mmlu/continuation/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,196 copying lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,198 copying lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,200 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,202 copying lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,204 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,206 copying lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,208 copying lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,210 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,212 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,214 copying lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,216 copying lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,218 copying lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,220 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,222 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,224 copying lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,226 copying lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,228 copying lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,230 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,232 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,235 copying lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,237 copying lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,239 copying lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,241 copying lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,243 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,245 copying lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,247 copying lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,249 copying lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,251 copying lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,253 copying lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,255 copying lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,257 copying lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,259 copying lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,261 copying lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,263 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,265 copying lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,267 copying lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,269 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,271 copying lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,274 copying lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:32,275 creating build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,276 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,279 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,281 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,283 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,286 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,288 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,290 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,292 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,295 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,297 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,299 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,302 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,304 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,306 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,308 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,311 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,313 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,316 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,318 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,320 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,322 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,325 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,327 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,330 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,332 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,334 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,337 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,339 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,342 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,344 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,347 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,349 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,351 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,354 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,356 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,358 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,361 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,363 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,366 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,368 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,370 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,373 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,375 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,377 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,379 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,382 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,385 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,387 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,390 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,393 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,396 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,399 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,401 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,404 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,407 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,410 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,413 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,416 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:32,418 creating build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,420 copying lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,423 copying lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,425 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,428 copying lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,431 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,434 copying lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,437 copying lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,440 copying lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,443 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,446 copying lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,450 copying lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,452 copying lm_eval/tasks/mmlu/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,455 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,458 copying lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,460 copying lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,463 copying lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,465 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,468 copying lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,470 copying lm_eval/tasks/mmlu/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,473 copying lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,476 copying lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,478 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,481 copying lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,483 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,486 copying lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,488 copying lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,490 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,492 copying lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,494 copying lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,497 copying lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,499 copying lm_eval/tasks/mmlu/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,501 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,503 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,505 copying lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,507 copying lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,509 copying lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,511 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,513 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,515 copying lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,517 copying lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,519 copying lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,521 copying lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,524 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,526 copying lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,528 copying lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,530 copying lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,532 copying lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,533 copying lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,535 copying lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,537 copying lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,539 copying lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,541 copying lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,543 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,545 copying lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,547 copying lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,550 copying lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,552 copying lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,554 copying lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:32,556 creating build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,557 copying lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,559 copying lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,561 copying lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,563 copying lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,565 copying lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,567 copying lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,569 copying lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,571 copying lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,573 copying lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,575 copying lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,577 copying lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,579 copying lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,581 copying lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,583 copying lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,585 copying lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,587 copying lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,589 copying lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,591 copying lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,593 copying lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,595 copying lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,597 copying lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,599 copying lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,600 copying lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,602 copying lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,604 copying lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,606 copying lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,608 copying lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,610 copying lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,612 copying lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,615 copying lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,617 copying lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,619 copying lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,620 copying lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,623 copying lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,625 copying lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,627 copying lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,629 copying lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,631 copying lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,633 copying lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,635 copying lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,637 copying lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,639 copying lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,641 copying lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,643 copying lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,645 copying lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,647 copying lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,649 copying lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,651 copying lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,653 copying lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,655 copying lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,656 copying lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,658 copying lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,660 copying lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,662 copying lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,664 copying lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,666 copying lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,668 copying lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,670 copying lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:32,672 copying lm_eval/tasks/wikitext/wikitext.yaml -> build/lib/lm_eval/tasks/wikitext 2024-07-01T14:47:32,674 creating build/lib/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:32,675 copying lm_eval/tasks/kormedmcqa/kormedmcqa_pharm.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:32,677 copying lm_eval/tasks/kormedmcqa/kormedmcqa_nurse.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:32,679 copying lm_eval/tasks/kormedmcqa/kormedmcqa_doctor.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:32,681 copying lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:32,683 copying lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:32,686 copying lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:32,688 copying lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:32,690 copying lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:32,692 copying lm_eval/tasks/eus_reading/eus_reading.yaml -> build/lib/lm_eval/tasks/eus_reading 2024-07-01T14:47:32,694 creating build/lib/lm_eval/tasks/commonsense_qa 2024-07-01T14:47:32,695 copying lm_eval/tasks/commonsense_qa/default.yaml -> build/lib/lm_eval/tasks/commonsense_qa 2024-07-01T14:47:32,697 creating build/lib/lm_eval/tasks/asdiv 2024-07-01T14:47:32,698 copying lm_eval/tasks/asdiv/default.yaml -> build/lib/lm_eval/tasks/asdiv 2024-07-01T14:47:32,700 creating build/lib/lm_eval/tasks/fld 2024-07-01T14:47:32,701 copying lm_eval/tasks/fld/fld_default.yaml -> build/lib/lm_eval/tasks/fld 2024-07-01T14:47:32,703 copying lm_eval/tasks/fld/fld_logical_formula_default.yaml -> build/lib/lm_eval/tasks/fld 2024-07-01T14:47:32,705 copying lm_eval/tasks/fld/fld_star.yaml -> build/lib/lm_eval/tasks/fld 2024-07-01T14:47:32,707 copying lm_eval/tasks/fld/fld_logical_formula_star.yaml -> build/lib/lm_eval/tasks/fld 2024-07-01T14:47:32,709 creating build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,710 copying lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,712 copying lm_eval/tasks/pile/pile_uspto.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,715 copying lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,717 copying lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,719 copying lm_eval/tasks/pile/pile_pile-cc.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,721 copying lm_eval/tasks/pile/pile_philpapers.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,723 copying lm_eval/tasks/pile/pile_europarl.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,725 copying lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,727 copying lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,729 copying lm_eval/tasks/pile/pile_enron.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,731 copying lm_eval/tasks/pile/pile_github.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,733 copying lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,735 copying lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,737 copying lm_eval/tasks/pile/pile_stackexchange.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,739 copying lm_eval/tasks/pile/pile_books3.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,740 copying lm_eval/tasks/pile/pile_arxiv.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,743 copying lm_eval/tasks/pile/pile_gutenberg.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,745 copying lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,747 copying lm_eval/tasks/pile/pile_freelaw.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,749 copying lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,750 copying lm_eval/tasks/pile/pile_hackernews.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,752 copying lm_eval/tasks/pile/pile_wikipedia.yaml -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:32,754 creating build/lib/lm_eval/tasks/piqa 2024-07-01T14:47:32,755 copying lm_eval/tasks/piqa/piqa.yaml -> build/lib/lm_eval/tasks/piqa 2024-07-01T14:47:32,757 creating build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,758 copying lm_eval/tasks/xstorycloze/default_hi.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,760 copying lm_eval/tasks/xstorycloze/default_ar.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,763 copying lm_eval/tasks/xstorycloze/default_ru.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,765 copying lm_eval/tasks/xstorycloze/default_en.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,767 copying lm_eval/tasks/xstorycloze/default_zh.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,769 copying lm_eval/tasks/xstorycloze/default_sw.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,771 copying lm_eval/tasks/xstorycloze/default_eu.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,773 copying lm_eval/tasks/xstorycloze/default_id.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,775 copying lm_eval/tasks/xstorycloze/default_te.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,777 copying lm_eval/tasks/xstorycloze/default_my.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,778 copying lm_eval/tasks/xstorycloze/default_es.yaml -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:32,780 creating build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,781 copying lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,784 copying lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,786 copying lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,788 copying lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,790 copying lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,792 copying lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,794 copying lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,796 copying lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,798 copying lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,800 copying lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:32,802 creating build/lib/lm_eval/tasks/prost 2024-07-01T14:47:32,803 copying lm_eval/tasks/prost/corypaik_prost.yaml -> build/lib/lm_eval/tasks/prost 2024-07-01T14:47:32,805 copying lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:32,807 copying lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:32,809 copying lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:32,811 copying lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:32,813 copying lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:32,815 copying lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:32,816 creating build/lib/lm_eval/tasks/triviaqa 2024-07-01T14:47:32,817 copying lm_eval/tasks/triviaqa/default.yaml -> build/lib/lm_eval/tasks/triviaqa 2024-07-01T14:47:32,819 copying lm_eval/tasks/hellaswag/hellaswag.yaml -> build/lib/lm_eval/tasks/hellaswag 2024-07-01T14:47:32,822 copying lm_eval/tasks/fda/fda.yaml -> build/lib/lm_eval/tasks/fda 2024-07-01T14:47:32,823 creating build/lib/lm_eval/tasks/arc 2024-07-01T14:47:32,824 copying lm_eval/tasks/arc/arc_easy.yaml -> build/lib/lm_eval/tasks/arc 2024-07-01T14:47:32,826 copying lm_eval/tasks/arc/arc_challenge.yaml -> build/lib/lm_eval/tasks/arc 2024-07-01T14:47:32,828 copying lm_eval/tasks/xcopa/default_ta.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,830 copying lm_eval/tasks/xcopa/default_th.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,833 copying lm_eval/tasks/xcopa/default_ht.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,835 copying lm_eval/tasks/xcopa/default_vi.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,837 copying lm_eval/tasks/xcopa/default_zh.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,839 copying lm_eval/tasks/xcopa/default_sw.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,841 copying lm_eval/tasks/xcopa/default_tr.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,843 copying lm_eval/tasks/xcopa/default_id.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,845 copying lm_eval/tasks/xcopa/default_it.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,847 copying lm_eval/tasks/xcopa/default_qu.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,849 copying lm_eval/tasks/xcopa/default_et.yaml -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:32,851 copying lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,853 copying lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,855 copying lm_eval/tasks/hendrycks_math/hendrycks_math.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,857 copying lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,859 copying lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,861 copying lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,864 copying lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,866 copying lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:32,867 copying lm_eval/tasks/toxigen/toxigen.yaml -> build/lib/lm_eval/tasks/toxigen 2024-07-01T14:47:32,869 copying lm_eval/tasks/eus_trivia/eus_trivia.yaml -> build/lib/lm_eval/tasks/eus_trivia 2024-07-01T14:47:32,871 creating build/lib/lm_eval/tasks/unscramble 2024-07-01T14:47:32,872 copying lm_eval/tasks/unscramble/anagrams2.yaml -> build/lib/lm_eval/tasks/unscramble 2024-07-01T14:47:32,874 copying lm_eval/tasks/unscramble/reversed_words.yaml -> build/lib/lm_eval/tasks/unscramble 2024-07-01T14:47:32,876 copying lm_eval/tasks/unscramble/random_insertion.yaml -> build/lib/lm_eval/tasks/unscramble 2024-07-01T14:47:32,878 copying lm_eval/tasks/unscramble/anagrams1.yaml -> build/lib/lm_eval/tasks/unscramble 2024-07-01T14:47:32,880 copying lm_eval/tasks/unscramble/cycle_letters.yaml -> build/lib/lm_eval/tasks/unscramble 2024-07-01T14:47:32,881 creating build/lib/lm_eval/tasks/openbookqa 2024-07-01T14:47:32,882 copying lm_eval/tasks/openbookqa/openbookqa.yaml -> build/lib/lm_eval/tasks/openbookqa 2024-07-01T14:47:32,884 copying lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,886 copying lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,888 copying lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,890 copying lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,892 copying lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,894 copying lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,896 copying lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,898 copying lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,900 copying lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,902 copying lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,904 copying lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,906 copying lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,908 copying lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,910 copying lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,912 copying lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,914 copying lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,916 copying lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,918 copying lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,920 copying lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,922 copying lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,925 copying lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,927 copying lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,929 copying lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,931 copying lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,933 copying lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,935 copying lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,937 copying lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,939 copying lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,941 copying lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,943 copying lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,944 copying lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,946 copying lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,948 copying lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,950 copying lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,952 copying lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,954 copying lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,956 copying lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,958 copying lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,960 copying lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,962 copying lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,964 copying lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,966 copying lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,968 copying lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,970 copying lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,972 copying lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,974 copying lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,976 copying lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,978 copying lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,980 copying lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,982 copying lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,984 copying lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,986 copying lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:32,988 creating build/lib/lm_eval/tasks/wmdp 2024-07-01T14:47:32,990 copying lm_eval/tasks/wmdp/wmdp_bio.yaml -> build/lib/lm_eval/tasks/wmdp 2024-07-01T14:47:32,992 copying lm_eval/tasks/wmdp/wmdp_chem.yaml -> build/lib/lm_eval/tasks/wmdp 2024-07-01T14:47:32,994 copying lm_eval/tasks/wmdp/wmdp_cyber.yaml -> build/lib/lm_eval/tasks/wmdp 2024-07-01T14:47:32,996 copying lm_eval/tasks/coqa/default.yaml -> build/lib/lm_eval/tasks/coqa 2024-07-01T14:47:32,998 copying lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,000 copying lm_eval/tasks/french_bench/french_bench_xnli.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,002 copying lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,004 copying lm_eval/tasks/french_bench/french_bench_grammar.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,006 copying lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,008 copying lm_eval/tasks/french_bench/french_bench_hellaswag.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,010 copying lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,012 copying lm_eval/tasks/french_bench/french_bench_multifquad.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,015 copying lm_eval/tasks/french_bench/french_bench_vocab.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,016 copying lm_eval/tasks/french_bench/french_bench_trivia.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,019 copying lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,020 copying lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,023 copying lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,025 copying lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,027 copying lm_eval/tasks/french_bench/french_bench_reading_comp.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,029 copying lm_eval/tasks/french_bench/french_bench_fquadv2.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,031 copying lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,033 copying lm_eval/tasks/french_bench/french_bench_boolqa.yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:33,035 copying lm_eval/tasks/squadv2/squadv2.yaml -> build/lib/lm_eval/tasks/squadv2 2024-07-01T14:47:33,037 creating build/lib/lm_eval/tasks/alghafa 2024-07-01T14:47:33,037 creating build/lib/lm_eval/tasks/alghafa/copa_ar 2024-07-01T14:47:33,038 copying lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml -> build/lib/lm_eval/tasks/alghafa/copa_ar 2024-07-01T14:47:33,040 creating build/lib/lm_eval/tasks/alghafa/piqa_ar 2024-07-01T14:47:33,042 copying lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml -> build/lib/lm_eval/tasks/alghafa/piqa_ar 2024-07-01T14:47:33,044 copying lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,046 copying lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,048 copying lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,050 copying lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,052 copying lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,054 copying lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,056 copying lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,058 copying lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,060 copying lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,062 copying lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,064 copying lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,066 copying lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,068 copying lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,070 copying lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,072 copying lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,074 copying lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,076 copying lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,078 copying lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,080 copying lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,082 copying lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,084 copying lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,086 copying lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,087 copying lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,089 copying lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,092 copying lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,094 copying lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,096 copying lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,097 copying lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,099 copying lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,101 copying lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,104 copying lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,106 copying lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,108 copying lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,110 copying lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,112 copying lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,114 copying lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,116 copying lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,118 copying lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,120 copying lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,122 copying lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,124 copying lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,126 copying lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,128 copying lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,130 copying lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,132 copying lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,134 copying lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,136 copying lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,138 copying lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,140 copying lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,142 copying lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,144 copying lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,146 copying lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,147 copying lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,149 copying lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,151 copying lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,153 copying lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,155 copying lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,157 copying lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,159 copying lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,160 copying lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,162 copying lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,164 copying lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,167 copying lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,169 copying lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,170 copying lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,172 copying lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,174 copying lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,176 copying lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,179 copying lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,181 copying lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,183 copying lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,185 copying lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,187 copying lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,188 copying lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,191 copying lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,193 copying lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,195 copying lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,197 copying lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,199 copying lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,201 copying lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,203 copying lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,205 copying lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,208 copying lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,209 copying lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,211 copying lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,213 copying lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,215 copying lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,217 copying lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,219 copying lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,221 copying lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,223 copying lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,225 copying lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,227 copying lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,229 copying lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,231 copying lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,233 copying lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,234 copying lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,236 copying lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,239 copying lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,241 copying lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,243 copying lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,244 copying lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,246 copying lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,248 copying lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,251 copying lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,253 copying lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,255 copying lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,257 copying lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,259 copying lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,261 copying lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,263 copying lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,265 copying lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,267 copying lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,269 copying lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,271 copying lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,273 copying lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,275 copying lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,277 copying lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,279 copying lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,281 copying lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,283 copying lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,285 copying lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:33,287 creating build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,288 copying lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,290 copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,292 copying lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,293 copying lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,295 copying lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,297 copying lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,299 copying lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,301 copying lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,303 copying lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,305 copying lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,307 copying lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,309 copying lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,311 copying lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,313 copying lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,315 copying lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,317 copying lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,319 copying lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,321 copying lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,323 copying lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,325 copying lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,327 copying lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,329 copying lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,331 copying lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,333 copying lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,336 copying lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,337 copying lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,339 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,341 copying lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,343 copying lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,345 copying lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,347 copying lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,350 copying lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,352 copying lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,354 copying lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,356 copying lm_eval/tasks/bigbench/generate_until/color.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,358 copying lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,360 copying lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,362 copying lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,364 copying lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,366 copying lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,368 copying lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,370 copying lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,372 copying lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,374 copying lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,376 copying lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,379 copying lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,381 copying lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,383 copying lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,385 copying lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,387 copying lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,389 copying lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,391 copying lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,394 copying lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,396 copying lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,398 copying lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,400 copying lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,402 copying lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,404 copying lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,406 copying lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,408 copying lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,410 copying lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,412 copying lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,414 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,416 copying lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,418 copying lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,420 copying lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,422 copying lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,424 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,426 copying lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,428 copying lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,431 copying lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,433 copying lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,435 copying lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,437 copying lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,439 copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,441 copying lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,443 copying lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,446 copying lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,448 copying lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,450 copying lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,452 copying lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,454 copying lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,456 copying lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,458 copying lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,460 copying lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,462 copying lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,464 copying lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,466 copying lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,468 copying lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,470 copying lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,472 copying lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,474 copying lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,476 copying lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,479 copying lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,481 copying lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,483 copying lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,485 copying lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,487 copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,489 copying lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,491 copying lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,493 copying lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,495 copying lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,497 copying lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,499 copying lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,501 copying lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,504 copying lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,506 copying lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,508 copying lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,510 copying lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,512 copying lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,514 copying lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,516 copying lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,518 copying lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,520 copying lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,522 copying lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,524 copying lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,526 copying lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,528 copying lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,530 copying lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,532 copying lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,534 copying lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,536 copying lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,538 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,541 copying lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,543 copying lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,545 copying lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,547 copying lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,549 copying lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,551 copying lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,553 copying lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,555 copying lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,558 copying lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,560 copying lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,562 copying lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,564 copying lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,566 copying lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,568 copying lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,570 copying lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,572 copying lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,574 copying lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,576 copying lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,578 copying lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,580 copying lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,582 copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,584 copying lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,586 copying lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,588 copying lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,590 copying lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,592 copying lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,594 copying lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,596 copying lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,598 copying lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,600 copying lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,602 copying lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,604 copying lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,606 copying lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,608 copying lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,611 copying lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,613 copying lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,615 copying lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,617 copying lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,619 copying lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,621 copying lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,623 copying lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,626 copying lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,627 copying lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,629 copying lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:33,631 creating build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,632 copying lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,634 copying lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,636 copying lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,638 copying lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,640 copying lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,642 copying lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,644 copying lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,646 copying lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,648 copying lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,650 copying lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,652 copying lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,654 copying lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,656 copying lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,658 copying lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,660 copying lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,662 copying lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,664 copying lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,666 copying lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,668 copying lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,670 copying lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,672 copying lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,674 copying lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,676 copying lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,679 copying lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,680 copying lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,682 copying lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,684 copying lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,687 copying lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,688 copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,691 copying lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,693 copying lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,695 copying lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,697 copying lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,699 copying lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,701 copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,703 copying lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,705 copying lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,707 copying lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,709 copying lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,711 copying lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,713 copying lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,715 copying lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,717 copying lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,719 copying lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,722 copying lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,724 copying lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,726 copying lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,728 copying lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,730 copying lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,732 copying lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,734 copying lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,736 copying lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,739 copying lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,741 copying lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,743 copying lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,745 copying lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,747 copying lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,749 copying lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,751 copying lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,753 copying lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,755 copying lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,757 copying lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,759 copying lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,761 copying lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,763 copying lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,765 copying lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,767 copying lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,769 copying lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,771 copying lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,773 copying lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,775 copying lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,777 copying lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,779 copying lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,781 copying lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,783 copying lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,785 copying lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,787 copying lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,794 copying lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,796 copying lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,798 copying lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,800 copying lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,802 copying lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,804 copying lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,807 copying lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,809 copying lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,812 copying lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,814 copying lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,816 copying lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,818 copying lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,820 copying lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,822 copying lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,825 copying lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,827 copying lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,829 copying lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,831 copying lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,833 copying lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,835 copying lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,837 copying lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,838 copying lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,840 copying lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,843 copying lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,845 copying lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,847 copying lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,849 copying lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,851 copying lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,853 copying lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,855 copying lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,856 copying lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,858 copying lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,860 copying lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,862 copying lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,864 copying lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,866 copying lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,868 copying lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,870 copying lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,872 copying lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,874 copying lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,876 copying lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,878 copying lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:33,880 copying lm_eval/tasks/logiqa/logiqa.yaml -> build/lib/lm_eval/tasks/logiqa 2024-07-01T14:47:33,882 copying lm_eval/tasks/glianorex/glianorex_en.yaml -> build/lib/lm_eval/tasks/glianorex 2024-07-01T14:47:33,884 copying lm_eval/tasks/glianorex/glianorex.yaml -> build/lib/lm_eval/tasks/glianorex 2024-07-01T14:47:33,886 copying lm_eval/tasks/glianorex/glianorex_fr.yaml -> build/lib/lm_eval/tasks/glianorex 2024-07-01T14:47:33,888 copying lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,890 copying lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,892 copying lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,894 copying lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,896 copying lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,898 copying lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,900 copying lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,902 copying lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,904 copying lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,906 copying lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,908 copying lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,910 copying lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,912 copying lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,914 copying lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,916 copying lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,918 copying lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,920 copying lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,922 copying lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,924 copying lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,926 copying lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,928 copying lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,930 copying lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:33,931 creating build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,932 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,935 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,937 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,939 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,941 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,943 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,945 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:33,947 copying lm_eval/tasks/qasper/bool.yaml -> build/lib/lm_eval/tasks/qasper 2024-07-01T14:47:33,949 copying lm_eval/tasks/qasper/freeform.yaml -> build/lib/lm_eval/tasks/qasper 2024-07-01T14:47:33,951 copying lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:33,953 copying lm_eval/tasks/scrolls/scrolls_quality.yaml -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:33,955 copying lm_eval/tasks/scrolls/scrolls_contractnli.yaml -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:33,957 copying lm_eval/tasks/scrolls/scrolls_govreport.yaml -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:33,959 copying lm_eval/tasks/scrolls/scrolls_qmsum.yaml -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:33,961 copying lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:33,963 copying lm_eval/tasks/scrolls/scrolls_qasper.yaml -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:33,965 copying lm_eval/tasks/mathqa/mathqa.yaml -> build/lib/lm_eval/tasks/mathqa 2024-07-01T14:47:33,967 creating build/lib/lm_eval/tasks/storycloze 2024-07-01T14:47:33,968 copying lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/lib/lm_eval/tasks/storycloze 2024-07-01T14:47:33,970 copying lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/lib/lm_eval/tasks/storycloze 2024-07-01T14:47:33,972 copying lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,974 copying lm_eval/tasks/paloma/paloma_redpajama.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,976 copying lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,978 copying lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,980 copying lm_eval/tasks/paloma/paloma_mc4.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,982 copying lm_eval/tasks/paloma/paloma_c4_100_domains.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,984 copying lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,986 copying lm_eval/tasks/paloma/paloma_ptb.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,987 copying lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,989 copying lm_eval/tasks/paloma/paloma_c4_en.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,992 copying lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,994 copying lm_eval/tasks/paloma/paloma_gab.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,995 copying lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,997 copying lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:33,999 copying lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:34,001 copying lm_eval/tasks/paloma/paloma_wikitext_103.yaml -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:34,003 copying lm_eval/tasks/agieval/math.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,005 copying lm_eval/tasks/agieval/gaokao-mathcloze.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,007 copying lm_eval/tasks/agieval/lsat-ar.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,010 copying lm_eval/tasks/agieval/gaokao-chinese.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,012 copying lm_eval/tasks/agieval/jec-qa-kd.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,014 copying lm_eval/tasks/agieval/gaokao-chemistry.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,016 copying lm_eval/tasks/agieval/gaokao-physics.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,018 copying lm_eval/tasks/agieval/logiqa-zh.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,020 copying lm_eval/tasks/agieval/gaokao-biology.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,022 copying lm_eval/tasks/agieval/jec-qa-ca.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,024 copying lm_eval/tasks/agieval/gaokao-mathqa.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,026 copying lm_eval/tasks/agieval/logiqa-en.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,028 copying lm_eval/tasks/agieval/gaokao-history.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,030 copying lm_eval/tasks/agieval/gaokao-geography.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,032 copying lm_eval/tasks/agieval/lsat-rc.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,035 copying lm_eval/tasks/agieval/sat-en-without-passage.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,037 copying lm_eval/tasks/agieval/sat-math.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,039 copying lm_eval/tasks/agieval/aqua-rat.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,041 copying lm_eval/tasks/agieval/lsat-lr.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,043 copying lm_eval/tasks/agieval/sat-en.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,046 copying lm_eval/tasks/agieval/gaokao-english.yaml -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:34,047 creating build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,048 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,050 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,052 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,054 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,056 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,058 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,060 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,062 copying lm_eval/tasks/bertaqa/bertaqa_en.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,064 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,066 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,068 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,070 copying lm_eval/tasks/bertaqa/bertaqa_eu.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,072 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,074 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,076 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,078 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:34,080 creating build/lib/lm_eval/tasks/nq_open 2024-07-01T14:47:34,081 copying lm_eval/tasks/nq_open/nq_open.yaml -> build/lib/lm_eval/tasks/nq_open 2024-07-01T14:47:34,083 copying lm_eval/tasks/winogrande/default.yaml -> build/lib/lm_eval/tasks/winogrande 2024-07-01T14:47:34,085 copying lm_eval/tasks/race/race.yaml -> build/lib/lm_eval/tasks/race 2024-07-01T14:47:34,087 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,089 copying lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,091 copying lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,094 copying lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,096 copying lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,098 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,100 copying lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,102 copying lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,105 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,107 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,109 copying lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,111 copying lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,113 copying lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,115 copying lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,117 copying lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,119 copying lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,121 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,123 copying lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,125 copying lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,126 copying lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,128 copying lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,130 copying lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,133 copying lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,135 copying lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,137 copying lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,139 copying lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,141 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:34,143 creating build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,144 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,146 copying lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,148 copying lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,150 copying lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,152 copying lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,154 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,157 copying lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,159 copying lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,161 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,163 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,165 copying lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,168 copying lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,170 copying lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,172 copying lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,175 copying lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,177 copying lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,179 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,181 copying lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,183 copying lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,185 copying lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,187 copying lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,190 copying lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,192 copying lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,194 copying lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,196 copying lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,198 copying lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,200 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:34,203 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,205 copying lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,207 copying lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,209 copying lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,211 copying lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,213 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,215 copying lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,217 copying lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,219 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,221 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,223 copying lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,226 copying lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,228 copying lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,230 copying lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,232 copying lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,234 copying lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,236 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,238 copying lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,240 copying lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,243 copying lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,245 copying lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,247 copying lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,249 copying lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,251 copying lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,253 copying lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,255 copying lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,257 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:34,259 creating build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,260 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,262 copying lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,264 copying lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,266 copying lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,268 copying lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,270 copying lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,272 copying lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,274 copying lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,276 copying lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,278 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,280 copying lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,283 copying lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,285 copying lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,287 copying lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,290 copying lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,292 copying lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,294 copying lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,296 copying lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,298 copying lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,300 copying lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,302 copying lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,304 copying lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,306 copying lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,308 copying lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,311 copying lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,313 copying lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,316 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:34,318 creating build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:34,319 copying lm_eval/tasks/aexams/aexams_Science.yaml -> build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:34,321 copying lm_eval/tasks/aexams/aexams_Biology.yaml -> build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:34,323 copying lm_eval/tasks/aexams/aexams_Social.yaml -> build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:34,325 copying lm_eval/tasks/aexams/aexams_Physics.yaml -> build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:34,327 copying lm_eval/tasks/aexams/aexams_IslamicStudies.yaml -> build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:34,328 creating build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,329 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,331 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,333 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,335 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,337 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,339 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,341 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,343 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,345 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,347 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,350 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:34,352 creating build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,353 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,355 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,357 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,359 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,361 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,363 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,365 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,367 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,370 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,372 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,374 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:34,376 creating build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,377 copying lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,379 copying lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,381 copying lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,383 copying lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,385 copying lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,387 copying lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,389 copying lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,391 copying lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,393 copying lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,395 copying lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,397 copying lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:34,399 copying lm_eval/tasks/aclue/aclue_reading_comprehension.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,401 copying lm_eval/tasks/aclue/aclue_ancient_literature.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,403 copying lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,405 copying lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,407 copying lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,409 copying lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,411 copying lm_eval/tasks/aclue/aclue_couplet_prediction.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,413 copying lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,415 copying lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,418 copying lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,420 copying lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,422 copying lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,424 copying lm_eval/tasks/aclue/aclue_ancient_medical.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,426 copying lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,428 copying lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:34,430 creating build/lib/lm_eval/tasks/glue/sst2 2024-07-01T14:47:34,431 copying lm_eval/tasks/glue/sst2/default.yaml -> build/lib/lm_eval/tasks/glue/sst2 2024-07-01T14:47:34,434 creating build/lib/lm_eval/tasks/glue/rte 2024-07-01T14:47:34,435 copying lm_eval/tasks/glue/rte/default.yaml -> build/lib/lm_eval/tasks/glue/rte 2024-07-01T14:47:34,437 creating build/lib/lm_eval/tasks/glue/wnli 2024-07-01T14:47:34,438 copying lm_eval/tasks/glue/wnli/default.yaml -> build/lib/lm_eval/tasks/glue/wnli 2024-07-01T14:47:34,440 creating build/lib/lm_eval/tasks/glue/qqp 2024-07-01T14:47:34,441 copying lm_eval/tasks/glue/qqp/default.yaml -> build/lib/lm_eval/tasks/glue/qqp 2024-07-01T14:47:34,444 copying lm_eval/tasks/glue/mnli/default.yaml -> build/lib/lm_eval/tasks/glue/mnli 2024-07-01T14:47:34,446 copying lm_eval/tasks/glue/mnli/mismatch.yaml -> build/lib/lm_eval/tasks/glue/mnli 2024-07-01T14:47:34,448 creating build/lib/lm_eval/tasks/glue/mrpc 2024-07-01T14:47:34,449 copying lm_eval/tasks/glue/mrpc/default.yaml -> build/lib/lm_eval/tasks/glue/mrpc 2024-07-01T14:47:34,451 creating build/lib/lm_eval/tasks/glue/qnli 2024-07-01T14:47:34,452 copying lm_eval/tasks/glue/qnli/default.yaml -> build/lib/lm_eval/tasks/glue/qnli 2024-07-01T14:47:34,454 creating build/lib/lm_eval/tasks/glue/cola 2024-07-01T14:47:34,455 copying lm_eval/tasks/glue/cola/default.yaml -> build/lib/lm_eval/tasks/glue/cola 2024-07-01T14:47:34,457 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,458 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,460 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,462 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,464 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,466 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,468 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,470 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,473 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,475 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,477 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,479 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,481 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,483 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,485 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,487 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,489 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,491 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,493 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,495 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,497 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,499 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,501 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,503 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,505 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,507 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,509 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,511 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,513 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,515 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,517 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,519 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,521 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,523 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,525 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,527 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,529 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,531 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,533 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,535 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,537 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,539 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,541 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,544 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,546 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,548 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,550 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,553 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,555 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,558 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,560 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,562 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,564 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,566 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,568 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,570 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,572 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,575 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,577 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,579 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,581 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,583 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:34,585 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,587 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,589 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,591 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,594 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,596 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,598 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,600 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,603 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,605 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,607 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,609 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,611 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,614 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,616 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,618 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,620 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,622 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,625 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,627 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,629 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,631 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,633 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,635 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,637 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,639 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,641 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,643 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,645 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,647 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:34,649 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,651 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,653 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,655 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,658 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,659 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,662 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,664 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,667 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,669 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,671 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,673 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,676 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,678 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,680 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,682 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,684 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,686 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,688 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,690 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,692 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,694 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,696 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,698 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,700 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,702 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,704 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,706 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,708 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,710 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,712 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,714 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,716 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,718 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:34,720 copying lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,723 copying lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,725 copying lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,727 copying lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,729 copying lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,731 copying lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,734 copying lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,736 copying lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,738 copying lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,740 copying lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,742 copying lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,744 copying lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,746 copying lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,748 copying lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,751 copying lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,753 copying lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,755 copying lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,757 copying lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,759 copying lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,761 copying lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,763 copying lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,765 copying lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,767 copying lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,769 copying lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,771 copying lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,773 copying lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,775 copying lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,777 copying lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,779 copying lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,781 copying lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,783 copying lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:34,785 creating build/lib/lm_eval/tasks/babi 2024-07-01T14:47:34,786 copying lm_eval/tasks/babi/babi.yaml -> build/lib/lm_eval/tasks/babi 2024-07-01T14:47:34,788 creating build/lib/lm_eval/tasks/swag 2024-07-01T14:47:34,789 copying lm_eval/tasks/swag/swag.yaml -> build/lib/lm_eval/tasks/swag 2024-07-01T14:47:34,791 copying lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/lib/lm_eval/tasks/truthfulqa 2024-07-01T14:47:34,794 copying lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/lib/lm_eval/tasks/truthfulqa 2024-07-01T14:47:34,796 copying lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/lib/lm_eval/tasks/truthfulqa 2024-07-01T14:47:34,798 copying lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:34,800 copying lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:34,803 copying lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:34,805 copying lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:34,807 copying lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:34,809 copying lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:34,811 copying lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,814 copying lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,816 copying lm_eval/tasks/translation/wmt16_de-en.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,818 copying lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,820 copying lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,822 copying lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,824 copying lm_eval/tasks/translation/wmt16_en-de.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,826 copying lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:34,828 copying lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/lib/lm_eval/tasks/qa4mre 2024-07-01T14:47:34,830 copying lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/lib/lm_eval/tasks/qa4mre 2024-07-01T14:47:34,832 copying lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/lib/lm_eval/tasks/qa4mre 2024-07-01T14:47:34,834 copying lm_eval/tasks/copal_id/standard.yaml -> build/lib/lm_eval/tasks/copal_id 2024-07-01T14:47:34,836 copying lm_eval/tasks/copal_id/colloquial.yaml -> build/lib/lm_eval/tasks/copal_id 2024-07-01T14:47:34,838 creating build/lib/lm_eval/tasks/lambada 2024-07-01T14:47:34,839 copying lm_eval/tasks/lambada/lambada_openai.yaml -> build/lib/lm_eval/tasks/lambada 2024-07-01T14:47:34,841 copying lm_eval/tasks/lambada/lambada_standard.yaml -> build/lib/lm_eval/tasks/lambada 2024-07-01T14:47:34,843 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,846 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,848 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,850 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,852 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,854 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,856 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,858 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,860 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,862 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,865 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,867 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,869 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,871 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,873 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,875 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,877 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,879 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,881 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,883 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,885 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,887 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,889 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,891 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,893 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,895 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,897 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,899 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,902 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,904 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,906 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,908 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,910 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,912 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,914 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,916 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,918 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,920 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,922 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,924 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,926 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,927 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,929 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,931 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,934 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,935 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,937 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,940 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,942 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:34,944 creating build/lib/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:34,945 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:34,947 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:34,949 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:34,951 copying lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,953 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,956 copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,958 copying lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,960 copying lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,962 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,964 copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,966 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,968 copying lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,970 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,972 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,974 copying lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,976 copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,978 copying lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,980 copying lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,982 copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,984 copying lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,986 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,988 copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,991 copying lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,993 copying lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,995 copying lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,997 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:34,999 copying lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,001 copying lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,003 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,005 copying lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,007 copying lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,010 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,011 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,014 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,015 copying lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,017 copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,019 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,021 copying lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,023 copying lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,025 copying lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,027 copying lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,029 copying lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,031 copying lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,033 copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,035 copying lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,037 copying lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,039 copying lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,041 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,043 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,046 copying lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,047 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,049 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,051 copying lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,054 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,056 copying lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,058 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,060 copying lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,062 copying lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,064 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,066 copying lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,068 copying lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,070 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,072 copying lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,074 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,076 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,078 copying lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,080 copying lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,082 copying lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,084 copying lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,086 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,088 copying lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,090 copying lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,092 copying lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,094 copying lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,096 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,098 copying lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,100 copying lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,102 copying lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,104 copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,107 copying lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,109 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,110 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,112 copying lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,114 copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,116 copying lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,118 copying lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,120 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,122 copying lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,124 copying lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,127 copying lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,129 copying lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,131 copying lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,133 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,135 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,137 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,139 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,141 copying lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,143 copying lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,145 copying lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,147 copying lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,149 copying lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,151 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,153 copying lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,155 copying lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,158 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,160 copying lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,161 copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,163 copying lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,166 copying lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,167 copying lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,169 copying lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,172 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,174 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,176 copying lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,178 copying lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,180 copying lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,182 copying lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,184 copying lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,186 copying lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,188 copying lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,190 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,192 copying lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,194 copying lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,196 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,198 copying lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,200 copying lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,202 copying lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,204 copying lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,206 copying lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,208 copying lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,210 copying lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,212 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,214 copying lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,216 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,218 copying lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,220 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,222 copying lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,224 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:35,226 creating build/lib/lm_eval/tasks/siqa 2024-07-01T14:47:35,227 copying lm_eval/tasks/siqa/siqa.yaml -> build/lib/lm_eval/tasks/siqa 2024-07-01T14:47:35,229 copying lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/lib/lm_eval/tasks/wmt2016 2024-07-01T14:47:35,231 copying lm_eval/tasks/drop/default.yaml -> build/lib/lm_eval/tasks/drop 2024-07-01T14:47:35,233 copying lm_eval/tasks/mutual/multual_plus.yaml -> build/lib/lm_eval/tasks/mutual 2024-07-01T14:47:35,235 copying lm_eval/tasks/mutual/mutual.yaml -> build/lib/lm_eval/tasks/mutual 2024-07-01T14:47:35,237 creating build/lib/lm_eval/tasks/eus_proficiency 2024-07-01T14:47:35,238 copying lm_eval/tasks/eus_proficiency/eus_proficiency.yaml -> build/lib/lm_eval/tasks/eus_proficiency 2024-07-01T14:47:35,240 creating build/lib/lm_eval/tasks/kmmlu 2024-07-01T14:47:35,240 creating build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,241 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,243 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,245 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,247 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,249 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,251 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,253 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,255 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,258 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,260 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,262 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,263 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,265 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,267 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,269 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,271 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,273 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,275 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,277 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,279 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,281 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,283 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,286 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,288 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,290 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,292 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,294 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,296 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,298 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,300 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,302 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,304 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,306 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,308 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,310 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,312 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,315 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,316 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,318 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,320 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,322 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,324 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,326 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,328 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,330 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:35,332 creating build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,333 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,335 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,338 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,340 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,344 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,346 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,349 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,351 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,354 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,357 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,359 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,362 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,365 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,367 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,370 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,373 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,375 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,378 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,381 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,383 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,386 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,388 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,390 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,393 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,395 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,398 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,400 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,404 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,406 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,409 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,411 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,414 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,416 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,419 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,422 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,424 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,427 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,429 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,432 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,434 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,437 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,439 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,442 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,445 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,448 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:35,450 creating build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,451 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,453 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,455 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,458 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,459 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,461 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,464 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,466 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,468 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,470 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,472 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,474 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,476 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,478 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,480 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,482 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,484 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,486 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,488 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,490 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,492 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,494 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,496 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,498 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,500 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,502 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,504 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,506 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,508 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,511 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,513 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,515 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,518 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,520 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,522 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,524 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,526 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,528 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,530 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,532 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,534 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,536 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,538 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,540 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,542 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:35,544 creating build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,545 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,547 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,549 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,551 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,553 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,555 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,557 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,559 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,561 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,563 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,565 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,567 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,570 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,572 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,574 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,576 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,578 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,580 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,582 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,584 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,586 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,588 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,590 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,592 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,594 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,596 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,598 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,600 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,602 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,604 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,606 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,608 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,610 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,612 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,614 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,616 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,618 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,620 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,622 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,624 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,627 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,629 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,631 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,633 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,635 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:35,637 creating build/lib/lm_eval/tasks/gsm8k 2024-07-01T14:47:35,638 copying lm_eval/tasks/gsm8k/gsm8k.yaml -> build/lib/lm_eval/tasks/gsm8k 2024-07-01T14:47:35,640 copying lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/lib/lm_eval/tasks/gsm8k 2024-07-01T14:47:35,642 copying lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/lib/lm_eval/tasks/gsm8k 2024-07-01T14:47:35,645 copying lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml -> build/lib/lm_eval/tasks/gsm8k 2024-07-01T14:47:35,647 copying lm_eval/tasks/swde/swde.yaml -> build/lib/lm_eval/tasks/swde 2024-07-01T14:47:35,649 copying lm_eval/tasks/noticia/noticia.yaml -> build/lib/lm_eval/tasks/noticia 2024-07-01T14:47:35,651 creating build/lib/lm_eval/tasks/pile_10k 2024-07-01T14:47:35,651 copying lm_eval/tasks/pile_10k/pile_10k.yaml -> build/lib/lm_eval/tasks/pile_10k 2024-07-01T14:47:35,654 creating build/lib/lm_eval/tasks/xnli_eu 2024-07-01T14:47:35,654 copying lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml -> build/lib/lm_eval/tasks/xnli_eu 2024-07-01T14:47:35,657 copying lm_eval/tasks/xnli_eu/xnli_eu_native.yaml -> build/lib/lm_eval/tasks/xnli_eu 2024-07-01T14:47:35,659 copying lm_eval/tasks/xnli_eu/xnli_eu.yaml -> build/lib/lm_eval/tasks/xnli_eu 2024-07-01T14:47:35,660 creating build/lib/lm_eval/tasks/polemo2 2024-07-01T14:47:35,661 copying lm_eval/tasks/polemo2/polemo2_in.yaml -> build/lib/lm_eval/tasks/polemo2 2024-07-01T14:47:35,663 copying lm_eval/tasks/polemo2/polemo2_out.yaml -> build/lib/lm_eval/tasks/polemo2 2024-07-01T14:47:35,665 creating build/lib/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:35,666 copying lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:35,668 copying lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:35,670 copying lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:35,672 copying lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:35,674 copying lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:35,676 creating build/lib/lm_eval/tasks/mc_taco 2024-07-01T14:47:35,677 copying lm_eval/tasks/mc_taco/default.yaml -> build/lib/lm_eval/tasks/mc_taco 2024-07-01T14:47:35,679 copying lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,682 copying lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,684 copying lm_eval/tasks/tinyBenchmarks/tinyArc.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,686 copying lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,688 copying lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,690 copying lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,692 copying lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,694 copying lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:35,696 copying lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:35,698 copying lm_eval/tasks/kobest/kobest_wic.yaml -> build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:35,700 copying lm_eval/tasks/kobest/kobest_boolq.yaml -> build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:35,702 copying lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:35,704 copying lm_eval/tasks/kobest/kobest_copa.yaml -> build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:35,706 copying lm_eval/tasks/xnli/xnli_en.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,708 copying lm_eval/tasks/xnli/xnli_zh.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,711 copying lm_eval/tasks/xnli/xnli_de.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,713 copying lm_eval/tasks/xnli/xnli_ar.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,715 copying lm_eval/tasks/xnli/xnli_ru.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,717 copying lm_eval/tasks/xnli/xnli_ur.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,719 copying lm_eval/tasks/xnli/xnli_es.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,721 copying lm_eval/tasks/xnli/xnli_vi.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,723 copying lm_eval/tasks/xnli/xnli_th.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,725 copying lm_eval/tasks/xnli/xnli_hi.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,727 copying lm_eval/tasks/xnli/xnli_bg.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,729 copying lm_eval/tasks/xnli/xnli_tr.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,731 copying lm_eval/tasks/xnli/xnli_sw.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,733 copying lm_eval/tasks/xnli/xnli_el.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,735 copying lm_eval/tasks/xnli/xnli_fr.yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:35,737 creating build/lib/lm_eval/tasks/super_glue/rte 2024-07-01T14:47:35,738 copying lm_eval/tasks/super_glue/rte/default.yaml -> build/lib/lm_eval/tasks/super_glue/rte 2024-07-01T14:47:35,740 copying lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/rte 2024-07-01T14:47:35,742 copying lm_eval/tasks/super_glue/record/default.yaml -> build/lib/lm_eval/tasks/super_glue/record 2024-07-01T14:47:35,744 copying lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/record 2024-07-01T14:47:35,746 creating build/lib/lm_eval/tasks/super_glue/wic 2024-07-01T14:47:35,747 copying lm_eval/tasks/super_glue/wic/default.yaml -> build/lib/lm_eval/tasks/super_glue/wic 2024-07-01T14:47:35,749 copying lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wic 2024-07-01T14:47:35,751 copying lm_eval/tasks/super_glue/copa/default.yaml -> build/lib/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:35,753 copying lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:35,755 copying lm_eval/tasks/super_glue/wsc/default.yaml -> build/lib/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:35,758 copying lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:35,760 copying lm_eval/tasks/super_glue/multirc/default.yaml -> build/lib/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:35,762 copying lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:35,764 copying lm_eval/tasks/super_glue/cb/default.yaml -> build/lib/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:35,766 copying lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:35,768 creating build/lib/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:35,769 copying lm_eval/tasks/super_glue/boolq/default.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:35,771 copying lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:35,777 copying lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:35,779 copying lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:35,781 copying lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:35,784 copying lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:35,786 copying lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:35,788 copying lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:35,790 copying lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:35,792 copying lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:35,794 copying lm_eval/tasks/paws-x/paws_en.yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:35,796 copying lm_eval/tasks/paws-x/paws_ja.yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:35,798 copying lm_eval/tasks/paws-x/paws_zh.yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:35,800 copying lm_eval/tasks/paws-x/paws_fr.yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:35,802 copying lm_eval/tasks/paws-x/paws_es.yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:35,804 copying lm_eval/tasks/paws-x/paws_ko.yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:35,806 copying lm_eval/tasks/paws-x/paws_de.yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:35,808 copying lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/lib/lm_eval/tasks/pubmedqa 2024-07-01T14:47:35,810 creating build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,811 copying lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,814 copying lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,816 copying lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,818 copying lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,820 copying lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,822 copying lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,824 copying lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,826 copying lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,828 copying lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,830 copying lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,832 copying lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,834 copying lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:35,836 copying lm_eval/tasks/ifeval/ifeval.yaml -> build/lib/lm_eval/tasks/ifeval 2024-07-01T14:47:35,838 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_biology.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,840 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_islamic_studies.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,842 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,845 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_social_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,847 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_computer_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,849 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_arabic_language.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,851 copying lm_eval/tasks/arabicmmlu/arabicmmlu.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,853 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_economics.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,856 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,858 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_physics.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,860 copying lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,862 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_history.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,864 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_math.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,866 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_civics.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,868 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_computer_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,870 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_history.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,872 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_geography.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,874 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_geography.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,876 copying lm_eval/tasks/arabicmmlu/arabicmmlu_prof_law.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,878 copying lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,880 copying lm_eval/tasks/arabicmmlu/arabicmmlu_univ_political_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,882 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_geography.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,883 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_general_knowledge.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,886 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_history.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,888 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,890 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_philosophy.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,892 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_arabic_language.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,894 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_islamic_studies.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,896 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_islamic_studies.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,898 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_economics.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,900 copying lm_eval/tasks/arabicmmlu/arabicmmlu_univ_economics.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,902 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_general_knowledge.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,904 copying lm_eval/tasks/arabicmmlu/arabicmmlu_univ_management.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,906 copying lm_eval/tasks/arabicmmlu/arabicmmlu_univ_computer_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,908 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_arabic_language.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,910 copying lm_eval/tasks/arabicmmlu/arabicmmlu_univ_accounting.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,912 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_social_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,914 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_computer_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,916 copying lm_eval/tasks/arabicmmlu/arabicmmlu_primary_natural_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,918 copying lm_eval/tasks/arabicmmlu/arabicmmlu_middle_natural_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,920 copying lm_eval/tasks/arabicmmlu/arabicmmlu_high_civics.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:35,922 creating build/lib/lm_eval/tasks/anli 2024-07-01T14:47:35,923 copying lm_eval/tasks/anli/anli_r1.yaml -> build/lib/lm_eval/tasks/anli 2024-07-01T14:47:35,926 copying lm_eval/tasks/anli/anli_r2.yaml -> build/lib/lm_eval/tasks/anli 2024-07-01T14:47:35,928 copying lm_eval/tasks/anli/anli_r3.yaml -> build/lib/lm_eval/tasks/anli 2024-07-01T14:47:35,930 copying lm_eval/tasks/logiqa2/logieval.yaml -> build/lib/lm_eval/tasks/logiqa2 2024-07-01T14:47:35,932 copying lm_eval/tasks/logiqa2/logiqa2.yaml -> build/lib/lm_eval/tasks/logiqa2 2024-07-01T14:47:35,934 copying lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,936 copying lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,938 copying lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,940 copying lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,941 copying lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,943 copying lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,945 copying lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,947 copying lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,949 copying lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,951 copying lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,953 copying lm_eval/tasks/blimp/drop_argument.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,955 copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,957 copying lm_eval/tasks/blimp/inchoative.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,959 copying lm_eval/tasks/blimp/adjunct_island.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,961 copying lm_eval/tasks/blimp/causative.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,963 copying lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,965 copying lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,967 copying lm_eval/tasks/blimp/only_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,969 copying lm_eval/tasks/blimp/intransitive.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,971 copying lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,973 copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,975 copying lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,977 copying lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,979 copying lm_eval/tasks/blimp/wh_island.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,982 copying lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,984 copying lm_eval/tasks/blimp/passive_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,986 copying lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,988 copying lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,990 copying lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,992 copying lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,994 copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,996 copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:35,998 copying lm_eval/tasks/blimp/passive_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,000 copying lm_eval/tasks/blimp/npi_present_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,002 copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,004 copying lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,006 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,008 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,010 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,012 copying lm_eval/tasks/blimp/npi_present_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,014 copying lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,016 copying lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,018 copying lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,019 copying lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,021 copying lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,023 copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,025 copying lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,027 copying lm_eval/tasks/blimp/complex_NP_island.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,029 copying lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,031 copying lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,033 copying lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,035 copying lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,037 copying lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,039 copying lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,041 copying lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,044 copying lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,046 copying lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,048 copying lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,050 copying lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,052 copying lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,054 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,056 copying lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,058 copying lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,060 copying lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,063 copying lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,065 copying lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,067 copying lm_eval/tasks/blimp/transitive.yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,069 creating build/lib/lm_eval/tasks/benchmarks 2024-07-01T14:47:36,070 copying lm_eval/tasks/benchmarks/pythia.yaml -> build/lib/lm_eval/tasks/benchmarks 2024-07-01T14:47:36,072 copying lm_eval/tasks/benchmarks/t0_eval.yaml -> build/lib/lm_eval/tasks/benchmarks 2024-07-01T14:47:36,074 copying lm_eval/tasks/benchmarks/minerva_math.yaml -> build/lib/lm_eval/tasks/benchmarks 2024-07-01T14:47:36,076 copying lm_eval/tasks/benchmarks/openllm.yaml -> build/lib/lm_eval/tasks/benchmarks 2024-07-01T14:47:36,077 creating build/lib/lm_eval/tasks/benchmarks/multimedqa 2024-07-01T14:47:36,078 copying lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/lib/lm_eval/tasks/benchmarks/multimedqa 2024-07-01T14:47:36,080 creating build/lib/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:36,081 copying lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:36,083 copying lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:36,086 creating build/lib/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:36,087 copying lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:36,089 copying lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:36,091 copying lm_eval/tasks/medqa/medqa.yaml -> build/lib/lm_eval/tasks/medqa 2024-07-01T14:47:36,093 copying lm_eval/tasks/README.md -> build/lib/lm_eval/tasks 2024-07-01T14:47:36,096 copying lm_eval/tasks/gpqa/README.md -> build/lib/lm_eval/tasks/gpqa 2024-07-01T14:47:36,098 copying lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:36,102 copying lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:36,104 copying lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:36,107 copying lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:36,111 copying lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:36,114 copying lm_eval/tasks/webqs/README.md -> build/lib/lm_eval/tasks/webqs 2024-07-01T14:47:36,116 copying lm_eval/tasks/tmmluplus/subject.tsv -> build/lib/lm_eval/tasks/tmmluplus 2024-07-01T14:47:36,118 copying lm_eval/tasks/tmmluplus/README.md -> build/lib/lm_eval/tasks/tmmluplus 2024-07-01T14:47:36,121 copying lm_eval/tasks/tmmluplus/default/_default_template_yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:36,124 copying lm_eval/tasks/unitxt/unitxt_tasks.summarization.abstractive -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,126 copying lm_eval/tasks/unitxt/unitxt_tasks.span_labeling.extraction -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,128 copying lm_eval/tasks/unitxt/unitxt_tasks.classification.multi_label -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,130 copying lm_eval/tasks/unitxt/unitxt_tasks.regression.two_texts -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,132 copying lm_eval/tasks/unitxt/unitxt_tasks.grammatical_error_correction -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,134 copying lm_eval/tasks/unitxt/unitxt_tasks.classification.multi_class -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,136 copying lm_eval/tasks/unitxt/README.md -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,139 copying lm_eval/tasks/unitxt/unitxt_tasks.qa.with_context.extractive -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,141 copying lm_eval/tasks/unitxt/unitxt_datasets -> build/lib/lm_eval/tasks/unitxt 2024-07-01T14:47:36,143 copying lm_eval/tasks/sciq/README.md -> build/lib/lm_eval/tasks/sciq 2024-07-01T14:47:36,146 copying lm_eval/tasks/headqa/README.md -> build/lib/lm_eval/tasks/headqa 2024-07-01T14:47:36,148 copying lm_eval/tasks/eus_exams/eus_exams -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:36,150 copying lm_eval/tasks/eus_exams/eus_exams_es -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:36,153 copying lm_eval/tasks/eus_exams/eus_exams_eu -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:36,155 copying lm_eval/tasks/eus_exams/README.md -> build/lib/lm_eval/tasks/eus_exams 2024-07-01T14:47:36,158 copying lm_eval/tasks/cmmlu/README.md -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:36,160 copying lm_eval/tasks/cmmlu/_default_template_yaml -> build/lib/lm_eval/tasks/cmmlu 2024-07-01T14:47:36,162 copying lm_eval/tasks/haerae/_default_haerae_yaml -> build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:36,164 copying lm_eval/tasks/haerae/README.md -> build/lib/lm_eval/tasks/haerae 2024-07-01T14:47:36,166 copying lm_eval/tasks/squad_completion/README.md -> build/lib/lm_eval/tasks/squad_completion 2024-07-01T14:47:36,169 copying lm_eval/tasks/wsc273/README.md -> build/lib/lm_eval/tasks/wsc273 2024-07-01T14:47:36,171 copying lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/lib/lm_eval/tasks/csatqa 2024-07-01T14:47:36,175 copying lm_eval/tasks/basqueglue/README.md -> build/lib/lm_eval/tasks/basqueglue 2024-07-01T14:47:36,177 copying lm_eval/tasks/eq_bench/README.md -> build/lib/lm_eval/tasks/eq_bench 2024-07-01T14:47:36,180 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:36,182 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:36,185 copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:36,187 copying lm_eval/tasks/mmlu/continuation/_continuation_template_yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:36,189 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:36,195 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:36,197 copying lm_eval/tasks/mmlu/generative/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:36,199 copying lm_eval/tasks/mmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu/default 2024-07-01T14:47:36,201 copying lm_eval/tasks/wikitext/README.md -> build/lib/lm_eval/tasks/wikitext 2024-07-01T14:47:36,203 copying lm_eval/tasks/kormedmcqa/README.md -> build/lib/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:36,206 copying lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:36,208 copying lm_eval/tasks/hendrycks_ethics/README.md -> build/lib/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:36,211 copying lm_eval/tasks/eus_reading/README.md -> build/lib/lm_eval/tasks/eus_reading 2024-07-01T14:47:36,213 copying lm_eval/tasks/commonsense_qa/README.md -> build/lib/lm_eval/tasks/commonsense_qa 2024-07-01T14:47:36,215 copying lm_eval/tasks/asdiv/README.md -> build/lib/lm_eval/tasks/asdiv 2024-07-01T14:47:36,217 copying lm_eval/tasks/fld/README.md -> build/lib/lm_eval/tasks/fld 2024-07-01T14:47:36,219 copying lm_eval/tasks/pile/README.md -> build/lib/lm_eval/tasks/pile 2024-07-01T14:47:36,222 copying lm_eval/tasks/piqa/README.md -> build/lib/lm_eval/tasks/piqa 2024-07-01T14:47:36,224 copying lm_eval/tasks/xstorycloze/README.md -> build/lib/lm_eval/tasks/xstorycloze 2024-07-01T14:47:36,226 copying lm_eval/tasks/arithmetic/README.md -> build/lib/lm_eval/tasks/arithmetic 2024-07-01T14:47:36,228 copying lm_eval/tasks/prost/README.md -> build/lib/lm_eval/tasks/prost 2024-07-01T14:47:36,231 copying lm_eval/tasks/xwinograd/README.md -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:36,233 copying lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/lib/lm_eval/tasks/xwinograd 2024-07-01T14:47:36,235 copying lm_eval/tasks/triviaqa/README.md -> build/lib/lm_eval/tasks/triviaqa 2024-07-01T14:47:36,238 copying lm_eval/tasks/hellaswag/README.md -> build/lib/lm_eval/tasks/hellaswag 2024-07-01T14:47:36,240 copying lm_eval/tasks/fda/README.md -> build/lib/lm_eval/tasks/fda 2024-07-01T14:47:36,242 copying lm_eval/tasks/arc/README.md -> build/lib/lm_eval/tasks/arc 2024-07-01T14:47:36,245 copying lm_eval/tasks/xcopa/README.md -> build/lib/lm_eval/tasks/xcopa 2024-07-01T14:47:36,248 copying lm_eval/tasks/hendrycks_math/README.md -> build/lib/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:36,250 copying lm_eval/tasks/toxigen/README.md -> build/lib/lm_eval/tasks/toxigen 2024-07-01T14:47:36,253 copying lm_eval/tasks/eus_trivia/README.md -> build/lib/lm_eval/tasks/eus_trivia 2024-07-01T14:47:36,255 copying lm_eval/tasks/unscramble/README.md -> build/lib/lm_eval/tasks/unscramble 2024-07-01T14:47:36,257 copying lm_eval/tasks/openbookqa/README.md -> build/lib/lm_eval/tasks/openbookqa 2024-07-01T14:47:36,259 copying lm_eval/tasks/ceval/_default_ceval_yaml -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:36,262 copying lm_eval/tasks/ceval/README.md -> build/lib/lm_eval/tasks/ceval 2024-07-01T14:47:36,265 copying lm_eval/tasks/wmdp/README.md -> build/lib/lm_eval/tasks/wmdp 2024-07-01T14:47:36,267 copying lm_eval/tasks/wmdp/_default_template_yaml -> build/lib/lm_eval/tasks/wmdp 2024-07-01T14:47:36,269 copying lm_eval/tasks/coqa/README.md -> build/lib/lm_eval/tasks/coqa 2024-07-01T14:47:36,272 copying lm_eval/tasks/french_bench/README.md -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:36,275 copying lm_eval/tasks/french_bench/_default_template_yaml -> build/lib/lm_eval/tasks/french_bench 2024-07-01T14:47:36,277 copying lm_eval/tasks/squadv2/README.md -> build/lib/lm_eval/tasks/squadv2 2024-07-01T14:47:36,279 copying lm_eval/tasks/alghafa/copa_ar/README.md -> build/lib/lm_eval/tasks/alghafa/copa_ar 2024-07-01T14:47:36,281 copying lm_eval/tasks/alghafa/piqa_ar/README.md -> build/lib/lm_eval/tasks/alghafa/piqa_ar 2024-07-01T14:47:36,284 copying lm_eval/tasks/belebele/README.md -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:36,286 copying lm_eval/tasks/belebele/_default_template_yaml -> build/lib/lm_eval/tasks/belebele 2024-07-01T14:47:36,290 copying lm_eval/tasks/bigbench/multiple_choice_template_b_yaml -> build/lib/lm_eval/tasks/bigbench 2024-07-01T14:47:36,292 copying lm_eval/tasks/bigbench/multiple_choice_template_a_yaml -> build/lib/lm_eval/tasks/bigbench 2024-07-01T14:47:36,294 copying lm_eval/tasks/bigbench/generate_until_template_yaml -> build/lib/lm_eval/tasks/bigbench 2024-07-01T14:47:36,296 copying lm_eval/tasks/bigbench/README.md -> build/lib/lm_eval/tasks/bigbench 2024-07-01T14:47:36,299 copying lm_eval/tasks/logiqa/README.md -> build/lib/lm_eval/tasks/logiqa 2024-07-01T14:47:36,302 copying lm_eval/tasks/glianorex/README.md -> build/lib/lm_eval/tasks/glianorex 2024-07-01T14:47:36,304 copying lm_eval/tasks/crows_pairs/README.md -> build/lib/lm_eval/tasks/crows_pairs 2024-07-01T14:47:36,307 copying lm_eval/tasks/lambada_multilingual_stablelm/README.md -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:36,310 copying lm_eval/tasks/qasper/README.md -> build/lib/lm_eval/tasks/qasper 2024-07-01T14:47:36,313 copying lm_eval/tasks/scrolls/README.md -> build/lib/lm_eval/tasks/scrolls 2024-07-01T14:47:36,315 copying lm_eval/tasks/mathqa/README.md -> build/lib/lm_eval/tasks/mathqa 2024-07-01T14:47:36,317 copying lm_eval/tasks/storycloze/README.md -> build/lib/lm_eval/tasks/storycloze 2024-07-01T14:47:36,320 copying lm_eval/tasks/paloma/README.md -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:36,322 copying lm_eval/tasks/paloma/_paloma_template -> build/lib/lm_eval/tasks/paloma 2024-07-01T14:47:36,324 copying lm_eval/tasks/agieval/README.md -> build/lib/lm_eval/tasks/agieval 2024-07-01T14:47:36,327 copying lm_eval/tasks/bertaqa/_bertaqa_template -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:36,329 copying lm_eval/tasks/bertaqa/README.md -> build/lib/lm_eval/tasks/bertaqa 2024-07-01T14:47:36,331 copying lm_eval/tasks/nq_open/README.md -> build/lib/lm_eval/tasks/nq_open 2024-07-01T14:47:36,334 copying lm_eval/tasks/winogrande/README.md -> build/lib/lm_eval/tasks/winogrande 2024-07-01T14:47:36,337 copying lm_eval/tasks/race/README.md -> build/lib/lm_eval/tasks/race 2024-07-01T14:47:36,339 copying lm_eval/tasks/bbh/README.md -> build/lib/lm_eval/tasks/bbh 2024-07-01T14:47:36,342 copying lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:36,344 copying lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:36,346 copying lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:36,348 copying lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:36,350 copying lm_eval/tasks/aexams/README.md -> build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:36,352 copying lm_eval/tasks/aexams/_default_template_yaml -> build/lib/lm_eval/tasks/aexams 2024-07-01T14:47:36,355 copying lm_eval/tasks/mgsm/README.md -> build/lib/lm_eval/tasks/mgsm 2024-07-01T14:47:36,357 copying lm_eval/tasks/mgsm/gen_yaml.sh -> build/lib/lm_eval/tasks/mgsm 2024-07-01T14:47:36,359 copying lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:36,361 copying lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:36,363 copying lm_eval/tasks/mgsm/direct/direct_yaml -> build/lib/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:36,365 copying lm_eval/tasks/aclue/README.md -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:36,367 copying lm_eval/tasks/aclue/_default_template_yaml -> build/lib/lm_eval/tasks/aclue 2024-07-01T14:47:36,369 copying lm_eval/tasks/glue/README.md -> build/lib/lm_eval/tasks/glue 2024-07-01T14:47:36,372 copying lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:36,374 copying lm_eval/tasks/okapi/truthfulqa_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:36,377 copying lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:36,379 copying lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:36,381 copying lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:36,384 copying lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:36,386 copying lm_eval/tasks/okapi/arc_multilingual/_arc_yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:36,388 copying lm_eval/tasks/okapi/arc_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:36,390 copying lm_eval/tasks/babi/README.md -> build/lib/lm_eval/tasks/babi 2024-07-01T14:47:36,393 copying lm_eval/tasks/swag/README.md -> build/lib/lm_eval/tasks/swag 2024-07-01T14:47:36,395 copying lm_eval/tasks/truthfulqa/README.md -> build/lib/lm_eval/tasks/truthfulqa 2024-07-01T14:47:36,399 copying lm_eval/tasks/translation/wmt_common_yaml -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:36,401 copying lm_eval/tasks/translation/README.md -> build/lib/lm_eval/tasks/translation 2024-07-01T14:47:36,403 copying lm_eval/tasks/qa4mre/README.md -> build/lib/lm_eval/tasks/qa4mre 2024-07-01T14:47:36,406 copying lm_eval/tasks/copal_id/README.md -> build/lib/lm_eval/tasks/copal_id 2024-07-01T14:47:36,408 copying lm_eval/tasks/lambada/README.md -> build/lib/lm_eval/tasks/lambada 2024-07-01T14:47:36,411 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:36,413 creating build/lib/lm_eval/tasks/model_written_evals/winogenerated 2024-07-01T14:47:36,414 copying lm_eval/tasks/model_written_evals/winogenerated/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/winogenerated 2024-07-01T14:47:36,416 copying lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:36,418 copying lm_eval/tasks/siqa/README.md -> build/lib/lm_eval/tasks/siqa 2024-07-01T14:47:36,420 copying lm_eval/tasks/wmt2016/README.md -> build/lib/lm_eval/tasks/wmt2016 2024-07-01T14:47:36,423 copying lm_eval/tasks/drop/README.md -> build/lib/lm_eval/tasks/drop 2024-07-01T14:47:36,426 copying lm_eval/tasks/mutual/README.md -> build/lib/lm_eval/tasks/mutual 2024-07-01T14:47:36,428 copying lm_eval/tasks/eus_proficiency/README.md -> build/lib/lm_eval/tasks/eus_proficiency 2024-07-01T14:47:36,430 copying lm_eval/tasks/kmmlu/README.md -> build/lib/lm_eval/tasks/kmmlu 2024-07-01T14:47:36,432 copying lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:36,435 copying lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:36,437 copying lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:36,439 copying lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:36,441 copying lm_eval/tasks/gsm8k/README.md -> build/lib/lm_eval/tasks/gsm8k 2024-07-01T14:47:36,443 copying lm_eval/tasks/swde/README.md -> build/lib/lm_eval/tasks/swde 2024-07-01T14:47:36,447 copying lm_eval/tasks/noticia/README.md -> build/lib/lm_eval/tasks/noticia 2024-07-01T14:47:36,449 copying lm_eval/tasks/pile_10k/README.md -> build/lib/lm_eval/tasks/pile_10k 2024-07-01T14:47:36,451 copying lm_eval/tasks/xnli_eu/xnli_common_yaml -> build/lib/lm_eval/tasks/xnli_eu 2024-07-01T14:47:36,453 copying lm_eval/tasks/xnli_eu/README.md -> build/lib/lm_eval/tasks/xnli_eu 2024-07-01T14:47:36,455 copying lm_eval/tasks/polemo2/README.md -> build/lib/lm_eval/tasks/polemo2 2024-07-01T14:47:36,457 copying lm_eval/tasks/lambada_multilingual/README.md -> build/lib/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:36,459 copying lm_eval/tasks/mc_taco/README.md -> build/lib/lm_eval/tasks/mc_taco 2024-07-01T14:47:36,463 copying lm_eval/tasks/tinyBenchmarks/README.md -> build/lib/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:36,466 copying lm_eval/tasks/kobest/README.md -> build/lib/lm_eval/tasks/kobest 2024-07-01T14:47:36,468 copying lm_eval/tasks/xnli/xnli_common_yaml -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:36,471 copying lm_eval/tasks/xnli/README.md -> build/lib/lm_eval/tasks/xnli 2024-07-01T14:47:36,473 copying lm_eval/tasks/super_glue/README.md -> build/lib/lm_eval/tasks/super_glue 2024-07-01T14:47:36,480 copying lm_eval/tasks/minerva_math/README.md -> build/lib/lm_eval/tasks/minerva_math 2024-07-01T14:47:36,482 copying lm_eval/tasks/paws-x/pawsx_template_yaml -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:36,484 copying lm_eval/tasks/paws-x/README.md -> build/lib/lm_eval/tasks/paws-x 2024-07-01T14:47:36,487 copying lm_eval/tasks/pubmedqa/README.md -> build/lib/lm_eval/tasks/pubmedqa 2024-07-01T14:47:36,489 copying lm_eval/tasks/arc_mt/README.md -> build/lib/lm_eval/tasks/arc_mt 2024-07-01T14:47:36,492 copying lm_eval/tasks/ifeval/README.md -> build/lib/lm_eval/tasks/ifeval 2024-07-01T14:47:36,497 copying lm_eval/tasks/arabicmmlu/README.md -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:36,499 copying lm_eval/tasks/arabicmmlu/_default_template_yaml -> build/lib/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:36,501 copying lm_eval/tasks/anli/README.md -> build/lib/lm_eval/tasks/anli 2024-07-01T14:47:36,503 copying lm_eval/tasks/logiqa2/README.md -> build/lib/lm_eval/tasks/logiqa2 2024-07-01T14:47:36,506 copying lm_eval/tasks/blimp/_template_yaml -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,508 copying lm_eval/tasks/blimp/README.md -> build/lib/lm_eval/tasks/blimp 2024-07-01T14:47:36,510 copying lm_eval/tasks/benchmarks/multimedqa/README.md -> build/lib/lm_eval/tasks/benchmarks/multimedqa 2024-07-01T14:47:36,512 copying lm_eval/tasks/benchmarks/flan/_held_in_template_yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:36,515 copying lm_eval/tasks/lambada_cloze/README.md -> build/lib/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:37,735 installing to build/bdist.linux-armv7l/wheel 2024-07-01T14:47:37,736 running install 2024-07-01T14:47:37,761 running install_lib 2024-07-01T14:47:37,765 creating build/bdist.linux-armv7l 2024-07-01T14:47:37,766 creating build/bdist.linux-armv7l/wheel 2024-07-01T14:47:37,767 creating build/bdist.linux-armv7l/wheel/lm_eval 2024-07-01T14:47:37,768 creating build/bdist.linux-armv7l/wheel/lm_eval/decontamination 2024-07-01T14:47:37,769 copying build/lib/lm_eval/decontamination/archiver.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination 2024-07-01T14:47:37,771 copying build/lib/lm_eval/decontamination/janitor.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination 2024-07-01T14:47:37,774 copying build/lib/lm_eval/decontamination/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination 2024-07-01T14:47:37,775 copying build/lib/lm_eval/decontamination/decontaminate.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination 2024-07-01T14:47:37,778 copying build/lib/lm_eval/evaluator.py -> build/bdist.linux-armv7l/wheel/lm_eval 2024-07-01T14:47:37,781 creating build/bdist.linux-armv7l/wheel/lm_eval/prompts 2024-07-01T14:47:37,782 copying build/lib/lm_eval/prompts/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/prompts 2024-07-01T14:47:37,784 copying build/lib/lm_eval/evaluator_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval 2024-07-01T14:47:37,786 copying build/lib/lm_eval/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval 2024-07-01T14:47:37,791 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks 2024-07-01T14:47:37,792 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa 2024-07-01T14:47:37,794 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:37,795 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:37,797 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:37,799 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:37,800 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:37,802 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:37,804 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2024-07-01T14:47:37,806 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:37,807 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:37,809 copying build/lib/lm_eval/tasks/gpqa/n_shot/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:37,811 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:37,813 copying build/lib/lm_eval/tasks/gpqa/n_shot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:37,815 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:37,816 copying build/lib/lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2024-07-01T14:47:37,819 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:37,819 copying build/lib/lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:37,821 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:37,823 copying build/lib/lm_eval/tasks/gpqa/zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:37,825 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:37,826 copying build/lib/lm_eval/tasks/gpqa/zeroshot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:37,828 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2024-07-01T14:47:37,830 copying build/lib/lm_eval/tasks/gpqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa 2024-07-01T14:47:37,832 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:37,833 copying build/lib/lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:37,834 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:37,836 copying build/lib/lm_eval/tasks/gpqa/generative/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:37,838 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:37,840 copying build/lib/lm_eval/tasks/gpqa/generative/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:37,842 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2024-07-01T14:47:37,844 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:37,844 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:37,846 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:37,848 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:37,850 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:37,852 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:37,854 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2024-07-01T14:47:37,856 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs 2024-07-01T14:47:37,857 copying build/lib/lm_eval/tasks/webqs/webqs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs 2024-07-01T14:47:37,858 copying build/lib/lm_eval/tasks/webqs/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs 2024-07-01T14:47:37,860 copying build/lib/lm_eval/tasks/webqs/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs 2024-07-01T14:47:37,863 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus 2024-07-01T14:47:37,864 copying build/lib/lm_eval/tasks/tmmluplus/subject.tsv -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus 2024-07-01T14:47:37,865 copying build/lib/lm_eval/tasks/tmmluplus/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus 2024-07-01T14:47:37,868 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,869 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,871 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,873 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,874 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,876 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,877 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,879 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,881 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,882 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,884 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,886 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,887 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,889 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,890 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,892 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,894 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,895 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,897 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,899 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,900 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,902 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,904 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,905 copying build/lib/lm_eval/tasks/tmmluplus/default/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,907 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,908 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,910 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,912 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,914 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,915 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,917 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,918 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,920 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,922 copying build/lib/lm_eval/tasks/tmmluplus/default/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,924 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,926 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,928 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,930 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,931 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,933 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,935 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,937 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,938 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,940 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,942 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,944 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,946 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,947 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,949 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,950 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,952 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,954 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,956 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,957 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,959 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,961 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,962 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,964 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,966 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,967 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,969 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,971 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,973 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,975 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,976 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,978 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,980 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,981 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,983 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,985 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,987 copying build/lib/lm_eval/tasks/tmmluplus/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,988 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2024-07-01T14:47:37,991 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:37,991 copying build/lib/lm_eval/tasks/unitxt/unitxt_wrapper.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:37,993 copying build/lib/lm_eval/tasks/unitxt/20_newsgroups.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:37,995 copying build/lib/lm_eval/tasks/unitxt/cnn_dailymail.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:37,996 copying build/lib/lm_eval/tasks/unitxt/dbpedia_14.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:37,998 copying build/lib/lm_eval/tasks/unitxt/ag_news.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,000 copying build/lib/lm_eval/tasks/unitxt/medical_abstracts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,002 copying build/lib/lm_eval/tasks/unitxt/yahoo_answers_topics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,003 copying build/lib/lm_eval/tasks/unitxt/unfair_tos.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,005 copying build/lib/lm_eval/tasks/unitxt/unitxt_tasks.summarization.abstractive -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,007 copying build/lib/lm_eval/tasks/unitxt/unitxt_tasks.span_labeling.extraction -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,008 copying build/lib/lm_eval/tasks/unitxt/financial_tweets.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,010 copying build/lib/lm_eval/tasks/unitxt/generate_yamls.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,012 copying build/lib/lm_eval/tasks/unitxt/unitxt_tasks.classification.multi_label -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,014 copying build/lib/lm_eval/tasks/unitxt/unitxt_tasks.regression.two_texts -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,016 copying build/lib/lm_eval/tasks/unitxt/stsb.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,017 copying build/lib/lm_eval/tasks/unitxt/ethos_binary.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,019 copying build/lib/lm_eval/tasks/unitxt/unitxt_tasks.grammatical_error_correction -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,021 copying build/lib/lm_eval/tasks/unitxt/unitxt_tasks.classification.multi_class -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,022 copying build/lib/lm_eval/tasks/unitxt/xsum.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,024 copying build/lib/lm_eval/tasks/unitxt/banking77.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,026 copying build/lib/lm_eval/tasks/unitxt/law_stack_exchange.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,027 copying build/lib/lm_eval/tasks/unitxt/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,029 copying build/lib/lm_eval/tasks/unitxt/unitxt_tasks.qa.with_context.extractive -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,031 copying build/lib/lm_eval/tasks/unitxt/atis.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,033 copying build/lib/lm_eval/tasks/unitxt/coedit_gec.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,034 copying build/lib/lm_eval/tasks/unitxt/claim_stance_topic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,036 copying build/lib/lm_eval/tasks/unitxt/argument_topic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,037 copying build/lib/lm_eval/tasks/unitxt/unitxt_datasets -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,039 copying build/lib/lm_eval/tasks/unitxt/ledgar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2024-07-01T14:47:38,041 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq 2024-07-01T14:47:38,042 copying build/lib/lm_eval/tasks/sciq/sciq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq 2024-07-01T14:47:38,043 copying build/lib/lm_eval/tasks/sciq/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq 2024-07-01T14:47:38,046 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa 2024-07-01T14:47:38,046 copying build/lib/lm_eval/tasks/medmcqa/medmcqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa 2024-07-01T14:47:38,048 copying build/lib/lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa 2024-07-01T14:47:38,050 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts 2024-07-01T14:47:38,051 copying build/lib/lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts 2024-07-01T14:47:38,053 copying build/lib/lm_eval/tasks/realtoxicityprompts/metric.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts 2024-07-01T14:47:38,055 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa 2024-07-01T14:47:38,056 copying build/lib/lm_eval/tasks/headqa/headqa_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa 2024-07-01T14:47:38,058 copying build/lib/lm_eval/tasks/headqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa 2024-07-01T14:47:38,059 copying build/lib/lm_eval/tasks/headqa/headqa_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa 2024-07-01T14:47:38,062 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,063 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,065 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,067 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,068 copying build/lib/lm_eval/tasks/eus_exams/eus_exams -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,070 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,072 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,074 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,075 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,077 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,079 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,080 copying build/lib/lm_eval/tasks/eus_exams/configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,082 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,084 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,085 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,087 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,088 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,090 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,091 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,093 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,095 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,096 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,098 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,100 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,102 copying build/lib/lm_eval/tasks/eus_exams/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,103 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,105 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,107 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,108 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,110 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,111 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,113 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,115 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,117 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,118 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,120 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,122 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,123 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,125 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,127 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,128 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,130 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,132 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,133 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,135 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,136 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,138 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,139 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,141 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,143 copying build/lib/lm_eval/tasks/eus_exams/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,145 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,147 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,148 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,150 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,152 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,154 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,156 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,157 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,159 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,161 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,162 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,164 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,166 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,167 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,169 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,171 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,172 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,174 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,176 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2024-07-01T14:47:38,179 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,179 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,181 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_commonsense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,183 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,185 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,187 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_medical_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,188 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_logical.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,190 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_food_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,192 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,194 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,196 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,198 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,199 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,201 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,203 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,205 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,207 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,208 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,210 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,212 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,213 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,215 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,217 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,219 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,220 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,222 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_ancient_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,223 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,225 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,227 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_legal_and_moral_basis.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,228 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,230 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,232 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,234 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_ethnology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,236 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_modern_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,237 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,239 copying build/lib/lm_eval/tasks/cmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,241 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,243 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_civil_service_exam.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,245 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,247 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_journalism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,248 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_construction_project_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,250 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,252 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_engineering_hydrology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,254 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,255 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,257 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,259 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,260 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_actuarial_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,262 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,264 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_information_and_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,266 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,267 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,269 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,270 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,272 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_agronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,274 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,275 copying build/lib/lm_eval/tasks/cmmlu/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,277 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_arts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,279 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_driving_rule.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,280 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,282 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,284 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_food_culture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,285 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,287 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,289 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,290 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_marxist_theory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,292 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,294 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,296 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,297 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,299 copying build/lib/lm_eval/tasks/cmmlu/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2024-07-01T14:47:38,301 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,302 copying build/lib/lm_eval/tasks/haerae/haerae_lw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,304 copying build/lib/lm_eval/tasks/haerae/haerae_sn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,306 copying build/lib/lm_eval/tasks/haerae/haerae_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,307 copying build/lib/lm_eval/tasks/haerae/_default_haerae_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,309 copying build/lib/lm_eval/tasks/haerae/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,311 copying build/lib/lm_eval/tasks/haerae/haerae_rw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,313 copying build/lib/lm_eval/tasks/haerae/haerae_gk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2024-07-01T14:47:38,315 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/squad_completion 2024-07-01T14:47:38,316 copying build/lib/lm_eval/tasks/squad_completion/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squad_completion 2024-07-01T14:47:38,318 copying build/lib/lm_eval/tasks/squad_completion/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squad_completion 2024-07-01T14:47:38,319 copying build/lib/lm_eval/tasks/squad_completion/squad_completion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squad_completion 2024-07-01T14:47:38,321 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273 2024-07-01T14:47:38,322 copying build/lib/lm_eval/tasks/wsc273/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273 2024-07-01T14:47:38,324 copying build/lib/lm_eval/tasks/wsc273/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273 2024-07-01T14:47:38,326 copying build/lib/lm_eval/tasks/wsc273/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273 2024-07-01T14:47:38,328 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,329 copying build/lib/lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,330 copying build/lib/lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,332 copying build/lib/lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,334 copying build/lib/lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,335 copying build/lib/lm_eval/tasks/csatqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,337 copying build/lib/lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,338 copying build/lib/lm_eval/tasks/csatqa/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,340 copying build/lib/lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,342 copying build/lib/lm_eval/tasks/csatqa/csatqa_li.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2024-07-01T14:47:38,344 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,345 copying build/lib/lm_eval/tasks/basqueglue/vaxx.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,347 copying build/lib/lm_eval/tasks/basqueglue/qnli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,348 copying build/lib/lm_eval/tasks/basqueglue/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,350 copying build/lib/lm_eval/tasks/basqueglue/wic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,352 copying build/lib/lm_eval/tasks/basqueglue/coref.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,354 copying build/lib/lm_eval/tasks/basqueglue/bhtc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,355 copying build/lib/lm_eval/tasks/basqueglue/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,357 copying build/lib/lm_eval/tasks/basqueglue/bec.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2024-07-01T14:47:38,359 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench 2024-07-01T14:47:38,360 copying build/lib/lm_eval/tasks/eq_bench/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench 2024-07-01T14:47:38,362 copying build/lib/lm_eval/tasks/eq_bench/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench 2024-07-01T14:47:38,364 copying build/lib/lm_eval/tasks/eq_bench/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench 2024-07-01T14:47:38,366 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu 2024-07-01T14:47:38,368 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,369 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,371 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,373 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,375 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,376 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,378 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,379 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,381 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,383 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,384 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,386 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,388 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,389 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,391 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,393 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,394 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,397 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,398 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,400 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,402 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,403 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,405 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,407 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,408 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,410 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,412 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,414 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,415 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,417 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,419 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,420 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,422 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,424 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,425 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,427 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,429 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,430 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,432 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,434 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,436 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,438 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,440 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,441 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,443 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,445 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,447 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,449 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,450 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,452 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,454 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,456 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,457 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,459 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,461 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,463 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,465 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,467 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,468 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,470 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,472 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2024-07-01T14:47:38,474 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot 2024-07-01T14:47:38,477 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,478 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,480 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,482 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,483 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,485 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,487 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,489 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,490 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,492 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,494 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,495 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,497 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,499 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,501 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,502 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,504 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,506 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,508 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,510 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,511 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,513 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,515 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,517 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,518 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,520 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,522 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,524 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,526 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,527 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,529 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,530 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,532 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,534 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,535 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,537 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,539 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,540 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,542 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,544 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,546 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,548 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,549 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,551 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,553 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,555 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,557 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,559 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,560 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,562 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,564 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,565 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,567 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,568 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,570 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,572 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,573 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,575 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,577 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,578 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2024-07-01T14:47:38,581 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,582 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,584 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,586 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,588 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,590 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,591 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,593 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,595 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,597 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,598 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,600 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,601 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,603 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,605 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,606 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,608 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,610 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,611 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,613 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,615 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,616 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,618 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,620 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,621 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,623 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,625 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,627 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,628 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,630 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,632 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,634 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,635 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,637 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,638 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,640 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,642 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,643 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,645 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,646 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,648 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,650 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,651 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,653 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,654 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,656 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,658 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,659 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,661 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,663 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,665 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,667 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,668 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,670 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,672 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,673 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,675 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,676 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,678 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,680 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,681 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2024-07-01T14:47:38,684 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,685 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,687 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,688 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,690 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,692 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,693 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,695 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,697 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,698 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,700 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,702 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,704 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,705 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,707 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,709 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,711 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,712 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,714 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,716 copying build/lib/lm_eval/tasks/mmlu/continuation/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,717 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,719 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,721 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,722 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,724 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,726 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,727 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,729 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,731 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,733 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,734 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,736 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,738 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,740 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,741 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,743 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,745 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,747 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,749 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,750 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,752 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,753 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,755 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,756 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,758 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,760 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,762 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,763 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,765 copying build/lib/lm_eval/tasks/mmlu/continuation/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,766 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,768 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,770 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,771 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,773 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,774 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,776 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,778 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,780 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,781 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,783 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2024-07-01T14:47:38,785 copying build/lib/lm_eval/tasks/mmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu 2024-07-01T14:47:38,789 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,789 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,792 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,794 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,795 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,798 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,799 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,801 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,803 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,805 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,807 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,809 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,811 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,812 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,815 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,817 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,819 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,821 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,823 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,825 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,826 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,829 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,831 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,833 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,836 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,838 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,840 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,842 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,844 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,846 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,848 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,850 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,852 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,854 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,855 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,858 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,859 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,861 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,863 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,865 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,867 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,872 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,875 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,876 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,878 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,880 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,882 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,884 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,886 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,888 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,890 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,891 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,893 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,895 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,897 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,899 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,900 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,902 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,903 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,905 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,907 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2024-07-01T14:47:38,910 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,911 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,913 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,915 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,917 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,918 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,920 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,922 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,924 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,925 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,927 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,929 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,931 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,932 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,934 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,935 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,937 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,938 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,940 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,942 copying build/lib/lm_eval/tasks/mmlu/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,943 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,945 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,947 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,948 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,950 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,951 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,953 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,955 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,956 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,958 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,960 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,962 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,963 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,965 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,967 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,969 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,971 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,972 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,974 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,976 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,978 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,979 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,981 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,983 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,984 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,986 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,988 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,989 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,991 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,993 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,995 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,996 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:38,998 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,000 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,002 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,003 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,005 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,007 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,009 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,010 copying build/lib/lm_eval/tasks/mmlu/generative/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2024-07-01T14:47:39,014 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,015 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,017 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,018 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,020 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,022 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,023 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,025 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,027 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,028 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,030 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,032 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,034 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,035 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,037 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,038 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,040 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,042 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,043 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,045 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,047 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,049 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,050 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,052 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,054 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,056 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,058 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,059 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,061 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,062 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,064 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,066 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,067 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,069 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,070 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,072 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,073 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,075 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,077 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,078 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,080 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,082 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,083 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,085 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,087 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,088 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,090 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,092 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,094 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,095 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,097 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,099 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,101 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,102 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,104 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,106 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,107 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,109 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,110 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,112 copying build/lib/lm_eval/tasks/mmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2024-07-01T14:47:39,114 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext 2024-07-01T14:47:39,115 copying build/lib/lm_eval/tasks/wikitext/wikitext.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext 2024-07-01T14:47:39,117 copying build/lib/lm_eval/tasks/wikitext/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext 2024-07-01T14:47:39,119 copying build/lib/lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext 2024-07-01T14:47:39,121 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:39,122 copying build/lib/lm_eval/tasks/kormedmcqa/kormedmcqa_pharm.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:39,124 copying build/lib/lm_eval/tasks/kormedmcqa/kormedmcqa_nurse.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:39,125 copying build/lib/lm_eval/tasks/kormedmcqa/kormedmcqa_doctor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:39,127 copying build/lib/lm_eval/tasks/kormedmcqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kormedmcqa 2024-07-01T14:47:39,129 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,130 copying build/lib/lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,132 copying build/lib/lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,134 copying build/lib/lm_eval/tasks/hendrycks_ethics/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,135 copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,137 copying build/lib/lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,139 copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,141 copying build/lib/lm_eval/tasks/hendrycks_ethics/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,143 copying build/lib/lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2024-07-01T14:47:39,145 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_reading 2024-07-01T14:47:39,146 copying build/lib/lm_eval/tasks/eus_reading/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_reading 2024-07-01T14:47:39,148 copying build/lib/lm_eval/tasks/eus_reading/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_reading 2024-07-01T14:47:39,150 copying build/lib/lm_eval/tasks/eus_reading/eus_reading.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_reading 2024-07-01T14:47:39,151 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/commonsense_qa 2024-07-01T14:47:39,152 copying build/lib/lm_eval/tasks/commonsense_qa/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/commonsense_qa 2024-07-01T14:47:39,154 copying build/lib/lm_eval/tasks/commonsense_qa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/commonsense_qa 2024-07-01T14:47:39,156 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv 2024-07-01T14:47:39,157 copying build/lib/lm_eval/tasks/asdiv/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv 2024-07-01T14:47:39,159 copying build/lib/lm_eval/tasks/asdiv/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv 2024-07-01T14:47:39,161 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2024-07-01T14:47:39,162 copying build/lib/lm_eval/tasks/fld/fld_default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2024-07-01T14:47:39,164 copying build/lib/lm_eval/tasks/fld/fld_logical_formula_default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2024-07-01T14:47:39,166 copying build/lib/lm_eval/tasks/fld/fld_star.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2024-07-01T14:47:39,167 copying build/lib/lm_eval/tasks/fld/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2024-07-01T14:47:39,169 copying build/lib/lm_eval/tasks/fld/fld_logical_formula_star.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2024-07-01T14:47:39,171 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,172 copying build/lib/lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,174 copying build/lib/lm_eval/tasks/pile/pile_uspto.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,175 copying build/lib/lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,177 copying build/lib/lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,178 copying build/lib/lm_eval/tasks/pile/pile_pile-cc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,180 copying build/lib/lm_eval/tasks/pile/pile_philpapers.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,182 copying build/lib/lm_eval/tasks/pile/pile_europarl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,183 copying build/lib/lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,185 copying build/lib/lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,187 copying build/lib/lm_eval/tasks/pile/pile_enron.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,189 copying build/lib/lm_eval/tasks/pile/pile_github.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,190 copying build/lib/lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,192 copying build/lib/lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,194 copying build/lib/lm_eval/tasks/pile/pile_stackexchange.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,196 copying build/lib/lm_eval/tasks/pile/pile_books3.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,198 copying build/lib/lm_eval/tasks/pile/pile_arxiv.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,199 copying build/lib/lm_eval/tasks/pile/pile_gutenberg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,201 copying build/lib/lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,203 copying build/lib/lm_eval/tasks/pile/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,205 copying build/lib/lm_eval/tasks/pile/pile_freelaw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,206 copying build/lib/lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,208 copying build/lib/lm_eval/tasks/pile/pile_hackernews.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,210 copying build/lib/lm_eval/tasks/pile/pile_wikipedia.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2024-07-01T14:47:39,212 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa 2024-07-01T14:47:39,212 copying build/lib/lm_eval/tasks/piqa/piqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa 2024-07-01T14:47:39,214 copying build/lib/lm_eval/tasks/piqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa 2024-07-01T14:47:39,216 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,217 copying build/lib/lm_eval/tasks/xstorycloze/default_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,219 copying build/lib/lm_eval/tasks/xstorycloze/default_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,221 copying build/lib/lm_eval/tasks/xstorycloze/default_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,222 copying build/lib/lm_eval/tasks/xstorycloze/default_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,224 copying build/lib/lm_eval/tasks/xstorycloze/default_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,225 copying build/lib/lm_eval/tasks/xstorycloze/default_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,227 copying build/lib/lm_eval/tasks/xstorycloze/default_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,229 copying build/lib/lm_eval/tasks/xstorycloze/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,230 copying build/lib/lm_eval/tasks/xstorycloze/default_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,232 copying build/lib/lm_eval/tasks/xstorycloze/default_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,234 copying build/lib/lm_eval/tasks/xstorycloze/default_my.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,235 copying build/lib/lm_eval/tasks/xstorycloze/default_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2024-07-01T14:47:39,238 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,238 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,240 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,242 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,244 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,246 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,247 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,249 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,251 copying build/lib/lm_eval/tasks/arithmetic/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,253 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,254 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,256 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2024-07-01T14:47:39,258 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost 2024-07-01T14:47:39,259 copying build/lib/lm_eval/tasks/prost/corypaik_prost.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost 2024-07-01T14:47:39,261 copying build/lib/lm_eval/tasks/prost/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost 2024-07-01T14:47:39,263 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,264 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,266 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,267 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,269 copying build/lib/lm_eval/tasks/xwinograd/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,271 copying build/lib/lm_eval/tasks/xwinograd/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,273 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,274 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,276 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,278 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2024-07-01T14:47:39,280 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa 2024-07-01T14:47:39,281 copying build/lib/lm_eval/tasks/triviaqa/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa 2024-07-01T14:47:39,283 copying build/lib/lm_eval/tasks/triviaqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa 2024-07-01T14:47:39,285 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag 2024-07-01T14:47:39,286 copying build/lib/lm_eval/tasks/hellaswag/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag 2024-07-01T14:47:39,288 copying build/lib/lm_eval/tasks/hellaswag/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag 2024-07-01T14:47:39,290 copying build/lib/lm_eval/tasks/hellaswag/hellaswag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag 2024-07-01T14:47:39,292 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/fda 2024-07-01T14:47:39,293 copying build/lib/lm_eval/tasks/fda/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fda 2024-07-01T14:47:39,295 copying build/lib/lm_eval/tasks/fda/fda.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fda 2024-07-01T14:47:39,297 copying build/lib/lm_eval/tasks/fda/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fda 2024-07-01T14:47:39,299 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc 2024-07-01T14:47:39,300 copying build/lib/lm_eval/tasks/arc/arc_easy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc 2024-07-01T14:47:39,302 copying build/lib/lm_eval/tasks/arc/arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc 2024-07-01T14:47:39,304 copying build/lib/lm_eval/tasks/arc/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc 2024-07-01T14:47:39,307 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,308 copying build/lib/lm_eval/tasks/xcopa/default_ta.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,310 copying build/lib/lm_eval/tasks/xcopa/default_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,311 copying build/lib/lm_eval/tasks/xcopa/default_ht.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,313 copying build/lib/lm_eval/tasks/xcopa/default_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,315 copying build/lib/lm_eval/tasks/xcopa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,317 copying build/lib/lm_eval/tasks/xcopa/default_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,319 copying build/lib/lm_eval/tasks/xcopa/default_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,320 copying build/lib/lm_eval/tasks/xcopa/default_tr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,322 copying build/lib/lm_eval/tasks/xcopa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,324 copying build/lib/lm_eval/tasks/xcopa/default_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,326 copying build/lib/lm_eval/tasks/xcopa/default_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,327 copying build/lib/lm_eval/tasks/xcopa/default_qu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,329 copying build/lib/lm_eval/tasks/xcopa/default_et.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2024-07-01T14:47:39,331 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,332 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,334 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,335 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,337 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,338 copying build/lib/lm_eval/tasks/hendrycks_math/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,340 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,342 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,344 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,346 copying build/lib/lm_eval/tasks/hendrycks_math/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,348 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2024-07-01T14:47:39,350 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen 2024-07-01T14:47:39,351 copying build/lib/lm_eval/tasks/toxigen/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen 2024-07-01T14:47:39,353 copying build/lib/lm_eval/tasks/toxigen/toxigen.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen 2024-07-01T14:47:39,355 copying build/lib/lm_eval/tasks/toxigen/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen 2024-07-01T14:47:39,357 copying build/lib/lm_eval/tasks/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks 2024-07-01T14:47:39,360 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_trivia 2024-07-01T14:47:39,361 copying build/lib/lm_eval/tasks/eus_trivia/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_trivia 2024-07-01T14:47:39,363 copying build/lib/lm_eval/tasks/eus_trivia/eus_trivia.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_trivia 2024-07-01T14:47:39,364 copying build/lib/lm_eval/tasks/eus_trivia/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_trivia 2024-07-01T14:47:39,367 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2024-07-01T14:47:39,367 copying build/lib/lm_eval/tasks/unscramble/anagrams2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2024-07-01T14:47:39,369 copying build/lib/lm_eval/tasks/unscramble/reversed_words.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2024-07-01T14:47:39,371 copying build/lib/lm_eval/tasks/unscramble/random_insertion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2024-07-01T14:47:39,372 copying build/lib/lm_eval/tasks/unscramble/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2024-07-01T14:47:39,374 copying build/lib/lm_eval/tasks/unscramble/anagrams1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2024-07-01T14:47:39,376 copying build/lib/lm_eval/tasks/unscramble/cycle_letters.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2024-07-01T14:47:39,378 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa 2024-07-01T14:47:39,378 copying build/lib/lm_eval/tasks/openbookqa/openbookqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa 2024-07-01T14:47:39,380 copying build/lib/lm_eval/tasks/openbookqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa 2024-07-01T14:47:39,383 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,384 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,386 copying build/lib/lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,387 copying build/lib/lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,389 copying build/lib/lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,391 copying build/lib/lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,393 copying build/lib/lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,395 copying build/lib/lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,397 copying build/lib/lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,399 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,401 copying build/lib/lm_eval/tasks/ceval/_default_ceval_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,402 copying build/lib/lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,404 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,406 copying build/lib/lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,407 copying build/lib/lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,409 copying build/lib/lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,411 copying build/lib/lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,412 copying build/lib/lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,414 copying build/lib/lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,416 copying build/lib/lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,417 copying build/lib/lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,419 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,420 copying build/lib/lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,422 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,424 copying build/lib/lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,425 copying build/lib/lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,427 copying build/lib/lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,428 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,430 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,432 copying build/lib/lm_eval/tasks/ceval/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,434 copying build/lib/lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,436 copying build/lib/lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,438 copying build/lib/lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,439 copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,441 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,443 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,445 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,447 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,449 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,450 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,452 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,454 copying build/lib/lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,456 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,457 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,459 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,460 copying build/lib/lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,462 copying build/lib/lm_eval/tasks/ceval/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,464 copying build/lib/lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,465 copying build/lib/lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,467 copying build/lib/lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,468 copying build/lib/lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,470 copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,471 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,473 copying build/lib/lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,474 copying build/lib/lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,476 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2024-07-01T14:47:39,478 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2024-07-01T14:47:39,479 copying build/lib/lm_eval/tasks/wmdp/wmdp_bio.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2024-07-01T14:47:39,481 copying build/lib/lm_eval/tasks/wmdp/wmdp_chem.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2024-07-01T14:47:39,482 copying build/lib/lm_eval/tasks/wmdp/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2024-07-01T14:47:39,484 copying build/lib/lm_eval/tasks/wmdp/wmdp_cyber.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2024-07-01T14:47:39,486 copying build/lib/lm_eval/tasks/wmdp/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2024-07-01T14:47:39,488 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa 2024-07-01T14:47:39,489 copying build/lib/lm_eval/tasks/coqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa 2024-07-01T14:47:39,491 copying build/lib/lm_eval/tasks/coqa/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa 2024-07-01T14:47:39,492 copying build/lib/lm_eval/tasks/coqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa 2024-07-01T14:47:39,495 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,496 copying build/lib/lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,498 copying build/lib/lm_eval/tasks/french_bench/french_bench_xnli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,499 copying build/lib/lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,501 copying build/lib/lm_eval/tasks/french_bench/french_bench_grammar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,503 copying build/lib/lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,505 copying build/lib/lm_eval/tasks/french_bench/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,507 copying build/lib/lm_eval/tasks/french_bench/french_bench_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,508 copying build/lib/lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,510 copying build/lib/lm_eval/tasks/french_bench/french_bench_multifquad.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,511 copying build/lib/lm_eval/tasks/french_bench/french_bench_vocab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,513 copying build/lib/lm_eval/tasks/french_bench/french_bench_trivia.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,515 copying build/lib/lm_eval/tasks/french_bench/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,517 copying build/lib/lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,519 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,521 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,523 copying build/lib/lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,524 copying build/lib/lm_eval/tasks/french_bench/french_bench_reading_comp.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,526 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,528 copying build/lib/lm_eval/tasks/french_bench/preprocess_wikitext.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,530 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,532 copying build/lib/lm_eval/tasks/french_bench/french_bench_boolqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,534 copying build/lib/lm_eval/tasks/french_bench/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2024-07-01T14:47:39,536 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2 2024-07-01T14:47:39,537 copying build/lib/lm_eval/tasks/squadv2/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2 2024-07-01T14:47:39,539 copying build/lib/lm_eval/tasks/squadv2/squadv2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2 2024-07-01T14:47:39,541 copying build/lib/lm_eval/tasks/squadv2/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2 2024-07-01T14:47:39,543 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa 2024-07-01T14:47:39,545 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/copa_ar 2024-07-01T14:47:39,546 copying build/lib/lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/copa_ar 2024-07-01T14:47:39,548 copying build/lib/lm_eval/tasks/alghafa/copa_ar/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/copa_ar 2024-07-01T14:47:39,550 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/piqa_ar 2024-07-01T14:47:39,552 copying build/lib/lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/piqa_ar 2024-07-01T14:47:39,554 copying build/lib/lm_eval/tasks/alghafa/piqa_ar/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/piqa_ar 2024-07-01T14:47:39,558 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,559 copying build/lib/lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,561 copying build/lib/lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,563 copying build/lib/lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,564 copying build/lib/lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,566 copying build/lib/lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,567 copying build/lib/lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,569 copying build/lib/lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,571 copying build/lib/lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,572 copying build/lib/lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,574 copying build/lib/lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,576 copying build/lib/lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,577 copying build/lib/lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,579 copying build/lib/lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,580 copying build/lib/lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,582 copying build/lib/lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,584 copying build/lib/lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,585 copying build/lib/lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,587 copying build/lib/lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,589 copying build/lib/lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,591 copying build/lib/lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,594 copying build/lib/lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,595 copying build/lib/lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,597 copying build/lib/lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,599 copying build/lib/lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,600 copying build/lib/lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,602 copying build/lib/lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,604 copying build/lib/lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,606 copying build/lib/lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,608 copying build/lib/lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,610 copying build/lib/lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,611 copying build/lib/lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,613 copying build/lib/lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,616 copying build/lib/lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,618 copying build/lib/lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,619 copying build/lib/lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,621 copying build/lib/lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,622 copying build/lib/lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,624 copying build/lib/lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,625 copying build/lib/lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,627 copying build/lib/lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,629 copying build/lib/lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,630 copying build/lib/lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,632 copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,633 copying build/lib/lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,635 copying build/lib/lm_eval/tasks/belebele/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,637 copying build/lib/lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,638 copying build/lib/lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,640 copying build/lib/lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,642 copying build/lib/lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,644 copying build/lib/lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,645 copying build/lib/lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,647 copying build/lib/lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,649 copying build/lib/lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,651 copying build/lib/lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,652 copying build/lib/lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,654 copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,656 copying build/lib/lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,657 copying build/lib/lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,659 copying build/lib/lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,660 copying build/lib/lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,662 copying build/lib/lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,664 copying build/lib/lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,665 copying build/lib/lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,667 copying build/lib/lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,669 copying build/lib/lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,670 copying build/lib/lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,672 copying build/lib/lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,674 copying build/lib/lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,675 copying build/lib/lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,677 copying build/lib/lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,678 copying build/lib/lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,680 copying build/lib/lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,681 copying build/lib/lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,683 copying build/lib/lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,685 copying build/lib/lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,687 copying build/lib/lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,688 copying build/lib/lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,690 copying build/lib/lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,692 copying build/lib/lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,693 copying build/lib/lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,695 copying build/lib/lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,697 copying build/lib/lm_eval/tasks/belebele/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,699 copying build/lib/lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,700 copying build/lib/lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,702 copying build/lib/lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,703 copying build/lib/lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,705 copying build/lib/lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,706 copying build/lib/lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,708 copying build/lib/lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,710 copying build/lib/lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,711 copying build/lib/lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,713 copying build/lib/lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,714 copying build/lib/lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,716 copying build/lib/lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,718 copying build/lib/lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,719 copying build/lib/lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,721 copying build/lib/lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,723 copying build/lib/lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,725 copying build/lib/lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,726 copying build/lib/lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,728 copying build/lib/lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,730 copying build/lib/lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,731 copying build/lib/lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,733 copying build/lib/lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,735 copying build/lib/lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,737 copying build/lib/lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,738 copying build/lib/lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,740 copying build/lib/lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,742 copying build/lib/lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,744 copying build/lib/lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,745 copying build/lib/lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,747 copying build/lib/lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,749 copying build/lib/lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,750 copying build/lib/lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,752 copying build/lib/lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,754 copying build/lib/lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,755 copying build/lib/lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,757 copying build/lib/lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,758 copying build/lib/lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,760 copying build/lib/lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,761 copying build/lib/lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,763 copying build/lib/lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,765 copying build/lib/lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,766 copying build/lib/lm_eval/tasks/belebele/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,768 copying build/lib/lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2024-07-01T14:47:39,770 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2024-07-01T14:47:39,774 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,775 copying build/lib/lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,777 copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,779 copying build/lib/lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,781 copying build/lib/lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,782 copying build/lib/lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,784 copying build/lib/lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,786 copying build/lib/lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,787 copying build/lib/lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,789 copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,791 copying build/lib/lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,793 copying build/lib/lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,795 copying build/lib/lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,796 copying build/lib/lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,798 copying build/lib/lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,800 copying build/lib/lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,802 copying build/lib/lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,803 copying build/lib/lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,805 copying build/lib/lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,807 copying build/lib/lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,808 copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,810 copying build/lib/lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,812 copying build/lib/lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,814 copying build/lib/lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,815 copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,817 copying build/lib/lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,819 copying build/lib/lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,820 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,822 copying build/lib/lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,824 copying build/lib/lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,826 copying build/lib/lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,827 copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,829 copying build/lib/lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,831 copying build/lib/lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,833 copying build/lib/lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,835 copying build/lib/lm_eval/tasks/bigbench/generate_until/color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,836 copying build/lib/lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,838 copying build/lib/lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,840 copying build/lib/lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,841 copying build/lib/lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,843 copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,845 copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,861 copying build/lib/lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,863 copying build/lib/lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,864 copying build/lib/lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,866 copying build/lib/lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,868 copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,870 copying build/lib/lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,872 copying build/lib/lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,873 copying build/lib/lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,875 copying build/lib/lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,877 copying build/lib/lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,879 copying build/lib/lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,881 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,883 copying build/lib/lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,884 copying build/lib/lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,886 copying build/lib/lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,888 copying build/lib/lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,890 copying build/lib/lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,892 copying build/lib/lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,894 copying build/lib/lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,895 copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,897 copying build/lib/lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,898 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,900 copying build/lib/lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,902 copying build/lib/lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,904 copying build/lib/lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,905 copying build/lib/lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,907 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,909 copying build/lib/lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,910 copying build/lib/lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,912 copying build/lib/lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,914 copying build/lib/lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,916 copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,918 copying build/lib/lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,919 copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,921 copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,923 copying build/lib/lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,925 copying build/lib/lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,927 copying build/lib/lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,928 copying build/lib/lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,930 copying build/lib/lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,932 copying build/lib/lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,933 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,935 copying build/lib/lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,937 copying build/lib/lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,938 copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,940 copying build/lib/lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,942 copying build/lib/lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,943 copying build/lib/lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,945 copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,946 copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,948 copying build/lib/lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,950 copying build/lib/lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,952 copying build/lib/lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,954 copying build/lib/lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,955 copying build/lib/lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,957 copying build/lib/lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,959 copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,961 copying build/lib/lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,963 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,964 copying build/lib/lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,967 copying build/lib/lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,968 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,970 copying build/lib/lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,971 copying build/lib/lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,973 copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,975 copying build/lib/lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,976 copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,978 copying build/lib/lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,979 copying build/lib/lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,981 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,983 copying build/lib/lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,985 copying build/lib/lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,986 copying build/lib/lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,988 copying build/lib/lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,990 copying build/lib/lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,991 copying build/lib/lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,993 copying build/lib/lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,995 copying build/lib/lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,996 copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:39,998 copying build/lib/lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,000 copying build/lib/lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,001 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,003 copying build/lib/lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,005 copying build/lib/lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,007 copying build/lib/lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,008 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,010 copying build/lib/lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,012 copying build/lib/lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,013 copying build/lib/lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,015 copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,016 copying build/lib/lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,018 copying build/lib/lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,020 copying build/lib/lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,021 copying build/lib/lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,023 copying build/lib/lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,024 copying build/lib/lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,026 copying build/lib/lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,027 copying build/lib/lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,029 copying build/lib/lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,031 copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,033 copying build/lib/lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,034 copying build/lib/lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,036 copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,038 copying build/lib/lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,040 copying build/lib/lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,042 copying build/lib/lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,043 copying build/lib/lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,045 copying build/lib/lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,046 copying build/lib/lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,048 copying build/lib/lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,049 copying build/lib/lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,051 copying build/lib/lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,053 copying build/lib/lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,054 copying build/lib/lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,056 copying build/lib/lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,057 copying build/lib/lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,059 copying build/lib/lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,060 copying build/lib/lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,062 copying build/lib/lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,064 copying build/lib/lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,066 copying build/lib/lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,067 copying build/lib/lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,069 copying build/lib/lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,071 copying build/lib/lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,072 copying build/lib/lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,074 copying build/lib/lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2024-07-01T14:47:40,076 copying build/lib/lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2024-07-01T14:47:40,077 copying build/lib/lm_eval/tasks/bigbench/generate_tasks.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2024-07-01T14:47:40,080 copying build/lib/lm_eval/tasks/bigbench/multiple_choice_template_b_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2024-07-01T14:47:40,081 copying build/lib/lm_eval/tasks/bigbench/multiple_choice_template_a_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2024-07-01T14:47:40,083 copying build/lib/lm_eval/tasks/bigbench/generate_until_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2024-07-01T14:47:40,085 copying build/lib/lm_eval/tasks/bigbench/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2024-07-01T14:47:40,089 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,090 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,091 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,093 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,094 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,096 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,098 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,099 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,101 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,102 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,104 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,106 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,107 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,109 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,111 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,112 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,114 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,116 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,117 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,119 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,121 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,123 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,124 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,126 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,128 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,130 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,131 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,133 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,134 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,136 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,137 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,139 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,140 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,142 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,143 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,145 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,147 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,148 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,150 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,152 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,154 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,155 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,157 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,159 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,160 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,162 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,164 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,166 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,167 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,169 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,171 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,172 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,174 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,175 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,177 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,179 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,180 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,182 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,184 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,186 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,188 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,189 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,191 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,193 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,195 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,196 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,198 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,200 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,202 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,203 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,205 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,207 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,208 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,210 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,211 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,213 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,214 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,216 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,217 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,219 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,221 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,222 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,224 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,226 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,227 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,229 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,231 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,232 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,234 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,236 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,238 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,239 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,241 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,243 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,244 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,246 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,247 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,249 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,250 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,252 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,254 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,255 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,257 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,258 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,260 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,262 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,264 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,265 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,267 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,269 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,270 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,272 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,274 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,275 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,277 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,278 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,280 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,281 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,283 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,284 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2024-07-01T14:47:40,286 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa 2024-07-01T14:47:40,287 copying build/lib/lm_eval/tasks/logiqa/logiqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa 2024-07-01T14:47:40,289 copying build/lib/lm_eval/tasks/logiqa/utils_logiqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa 2024-07-01T14:47:40,291 copying build/lib/lm_eval/tasks/logiqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa 2024-07-01T14:47:40,293 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2024-07-01T14:47:40,294 copying build/lib/lm_eval/tasks/glianorex/preprocess_glianorex.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2024-07-01T14:47:40,295 copying build/lib/lm_eval/tasks/glianorex/glianorex_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2024-07-01T14:47:40,297 copying build/lib/lm_eval/tasks/glianorex/glianorex.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2024-07-01T14:47:40,299 copying build/lib/lm_eval/tasks/glianorex/glianorex_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2024-07-01T14:47:40,300 copying build/lib/lm_eval/tasks/glianorex/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2024-07-01T14:47:40,303 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,304 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,306 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,307 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,309 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,311 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,312 copying build/lib/lm_eval/tasks/crows_pairs/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,314 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,316 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,318 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,319 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,321 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,322 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,324 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,326 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,327 copying build/lib/lm_eval/tasks/crows_pairs/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,330 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,331 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,333 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,334 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,336 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,338 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,339 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,341 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,343 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2024-07-01T14:47:40,345 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,346 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,348 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,349 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,351 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,353 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,355 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,356 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,358 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2024-07-01T14:47:40,360 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2024-07-01T14:47:40,361 copying build/lib/lm_eval/tasks/qasper/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2024-07-01T14:47:40,363 copying build/lib/lm_eval/tasks/qasper/bool.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2024-07-01T14:47:40,365 copying build/lib/lm_eval/tasks/qasper/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2024-07-01T14:47:40,366 copying build/lib/lm_eval/tasks/qasper/freeform.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2024-07-01T14:47:40,368 copying build/lib/lm_eval/tasks/qasper/metrics.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2024-07-01T14:47:40,371 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,372 copying build/lib/lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,374 copying build/lib/lm_eval/tasks/scrolls/scrolls_quality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,376 copying build/lib/lm_eval/tasks/scrolls/scrolls_contractnli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,378 copying build/lib/lm_eval/tasks/scrolls/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,381 copying build/lib/lm_eval/tasks/scrolls/scrolls_govreport.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,384 copying build/lib/lm_eval/tasks/scrolls/scrolls_qmsum.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,386 copying build/lib/lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,388 copying build/lib/lm_eval/tasks/scrolls/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,390 copying build/lib/lm_eval/tasks/scrolls/scrolls_qasper.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2024-07-01T14:47:40,393 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa 2024-07-01T14:47:40,394 copying build/lib/lm_eval/tasks/mathqa/mathqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa 2024-07-01T14:47:40,396 copying build/lib/lm_eval/tasks/mathqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa 2024-07-01T14:47:40,399 copying build/lib/lm_eval/tasks/mathqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa 2024-07-01T14:47:40,401 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze 2024-07-01T14:47:40,402 copying build/lib/lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze 2024-07-01T14:47:40,405 copying build/lib/lm_eval/tasks/storycloze/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze 2024-07-01T14:47:40,407 copying build/lib/lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze 2024-07-01T14:47:40,410 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,411 copying build/lib/lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,413 copying build/lib/lm_eval/tasks/paloma/paloma_redpajama.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,416 copying build/lib/lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,419 copying build/lib/lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,421 copying build/lib/lm_eval/tasks/paloma/paloma_mc4.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,423 copying build/lib/lm_eval/tasks/paloma/paloma_c4_100_domains.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,425 copying build/lib/lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,427 copying build/lib/lm_eval/tasks/paloma/paloma_ptb.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,429 copying build/lib/lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,432 copying build/lib/lm_eval/tasks/paloma/paloma_c4_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,434 copying build/lib/lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,436 copying build/lib/lm_eval/tasks/paloma/paloma_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,438 copying build/lib/lm_eval/tasks/paloma/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,441 copying build/lib/lm_eval/tasks/paloma/paloma_gab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,443 copying build/lib/lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,445 copying build/lib/lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,447 copying build/lib/lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,449 copying build/lib/lm_eval/tasks/paloma/paloma_wikitext_103.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,451 copying build/lib/lm_eval/tasks/paloma/_paloma_template -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2024-07-01T14:47:40,454 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,455 copying build/lib/lm_eval/tasks/agieval/math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,458 copying build/lib/lm_eval/tasks/agieval/gaokao-mathcloze.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,460 copying build/lib/lm_eval/tasks/agieval/lsat-ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,462 copying build/lib/lm_eval/tasks/agieval/gaokao-chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,464 copying build/lib/lm_eval/tasks/agieval/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,467 copying build/lib/lm_eval/tasks/agieval/jec-qa-kd.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,469 copying build/lib/lm_eval/tasks/agieval/gaokao-chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,471 copying build/lib/lm_eval/tasks/agieval/gaokao-physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,472 copying build/lib/lm_eval/tasks/agieval/logiqa-zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,474 copying build/lib/lm_eval/tasks/agieval/gaokao-biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,476 copying build/lib/lm_eval/tasks/agieval/jec-qa-ca.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,477 copying build/lib/lm_eval/tasks/agieval/gaokao-mathqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,479 copying build/lib/lm_eval/tasks/agieval/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,482 copying build/lib/lm_eval/tasks/agieval/logiqa-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,483 copying build/lib/lm_eval/tasks/agieval/gaokao-history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,485 copying build/lib/lm_eval/tasks/agieval/gaokao-geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,487 copying build/lib/lm_eval/tasks/agieval/lsat-rc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,488 copying build/lib/lm_eval/tasks/agieval/sat-en-without-passage.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,490 copying build/lib/lm_eval/tasks/agieval/sat-math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,491 copying build/lib/lm_eval/tasks/agieval/aqua-rat.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,493 copying build/lib/lm_eval/tasks/agieval/lsat-lr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,495 copying build/lib/lm_eval/tasks/agieval/sat-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,496 copying build/lib/lm_eval/tasks/agieval/gaokao-english.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2024-07-01T14:47:40,499 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,500 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,501 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,503 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,504 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,506 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,508 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,509 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,511 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,513 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,515 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,516 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,518 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,520 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,522 copying build/lib/lm_eval/tasks/bertaqa/_bertaqa_template -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,524 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,526 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,527 copying build/lib/lm_eval/tasks/bertaqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,529 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2024-07-01T14:47:40,532 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open 2024-07-01T14:47:40,533 copying build/lib/lm_eval/tasks/nq_open/nq_open.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open 2024-07-01T14:47:40,535 copying build/lib/lm_eval/tasks/nq_open/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open 2024-07-01T14:47:40,537 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande 2024-07-01T14:47:40,538 copying build/lib/lm_eval/tasks/winogrande/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande 2024-07-01T14:47:40,539 copying build/lib/lm_eval/tasks/winogrande/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande 2024-07-01T14:47:40,541 copying build/lib/lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande 2024-07-01T14:47:40,544 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/race 2024-07-01T14:47:40,544 copying build/lib/lm_eval/tasks/race/race.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/race 2024-07-01T14:47:40,546 copying build/lib/lm_eval/tasks/race/preprocess_race.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/race 2024-07-01T14:47:40,548 copying build/lib/lm_eval/tasks/race/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/race 2024-07-01T14:47:40,551 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh 2024-07-01T14:47:40,553 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,554 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,556 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,558 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,560 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,562 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,563 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,565 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,567 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,569 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,571 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,572 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,575 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,576 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,578 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,581 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,583 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,585 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,587 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,589 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,590 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,592 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,594 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,596 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,598 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,600 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,601 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,603 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,605 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,607 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2024-07-01T14:47:40,609 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,610 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,612 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,614 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,617 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,618 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,620 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,623 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,625 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,627 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,629 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,631 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,634 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,636 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,638 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,640 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,642 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,644 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,646 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,647 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,649 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,651 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,654 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,655 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,657 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,659 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,661 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,663 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,665 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2024-07-01T14:47:40,667 copying build/lib/lm_eval/tasks/bbh/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh 2024-07-01T14:47:40,669 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,670 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,672 copying build/lib/lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,674 copying build/lib/lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,675 copying build/lib/lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,677 copying build/lib/lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,679 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,681 copying build/lib/lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,683 copying build/lib/lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,684 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,686 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,688 copying build/lib/lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,689 copying build/lib/lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,691 copying build/lib/lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,692 copying build/lib/lm_eval/tasks/bbh/zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,694 copying build/lib/lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,696 copying build/lib/lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,698 copying build/lib/lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,700 copying build/lib/lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,701 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,703 copying build/lib/lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,704 copying build/lib/lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,706 copying build/lib/lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,707 copying build/lib/lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,709 copying build/lib/lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,711 copying build/lib/lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,713 copying build/lib/lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,714 copying build/lib/lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,716 copying build/lib/lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,718 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2024-07-01T14:47:40,720 copying build/lib/lm_eval/tasks/bbh/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh 2024-07-01T14:47:40,723 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,724 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,726 copying build/lib/lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,728 copying build/lib/lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,730 copying build/lib/lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,732 copying build/lib/lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,733 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,735 copying build/lib/lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,737 copying build/lib/lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,738 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,740 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,742 copying build/lib/lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,744 copying build/lib/lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,746 copying build/lib/lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,748 copying build/lib/lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,750 copying build/lib/lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,752 copying build/lib/lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,754 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,755 copying build/lib/lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,757 copying build/lib/lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,759 copying build/lib/lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,761 copying build/lib/lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,763 copying build/lib/lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,764 copying build/lib/lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,766 copying build/lib/lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,768 copying build/lib/lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,770 copying build/lib/lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,772 copying build/lib/lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,774 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2024-07-01T14:47:40,776 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,777 copying build/lib/lm_eval/tasks/aexams/aexams_Science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,779 copying build/lib/lm_eval/tasks/aexams/aexams_Biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,781 copying build/lib/lm_eval/tasks/aexams/aexams_Social.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,782 copying build/lib/lm_eval/tasks/aexams/aexams_Physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,784 copying build/lib/lm_eval/tasks/aexams/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,785 copying build/lib/lm_eval/tasks/aexams/aexams_IslamicStudies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,787 copying build/lib/lm_eval/tasks/aexams/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2024-07-01T14:47:40,789 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm 2024-07-01T14:47:40,790 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,791 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,793 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,795 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,797 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,798 copying build/lib/lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,800 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,802 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,804 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,805 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,807 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,809 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,811 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2024-07-01T14:47:40,812 copying build/lib/lm_eval/tasks/mgsm/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm 2024-07-01T14:47:40,815 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,816 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,818 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,820 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,822 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,823 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,825 copying build/lib/lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,827 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,829 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,830 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,832 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,834 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,836 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2024-07-01T14:47:40,837 copying build/lib/lm_eval/tasks/mgsm/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm 2024-07-01T14:47:40,840 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,841 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,842 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,844 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,846 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,847 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,849 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,851 copying build/lib/lm_eval/tasks/mgsm/direct/direct_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,852 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,854 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,856 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,857 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,859 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2024-07-01T14:47:40,861 copying build/lib/lm_eval/tasks/mgsm/gen_yaml.sh -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm 2024-07-01T14:47:40,863 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,864 copying build/lib/lm_eval/tasks/aclue/aclue_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,866 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,868 copying build/lib/lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,869 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,871 copying build/lib/lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,873 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,875 copying build/lib/lm_eval/tasks/aclue/aclue_couplet_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,877 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,878 copying build/lib/lm_eval/tasks/aclue/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,880 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,882 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,884 copying build/lib/lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,885 copying build/lib/lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,887 copying build/lib/lm_eval/tasks/aclue/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,889 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_medical.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,891 copying build/lib/lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,892 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,894 copying build/lib/lm_eval/tasks/aclue/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2024-07-01T14:47:40,896 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue 2024-07-01T14:47:40,898 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/sst2 2024-07-01T14:47:40,899 copying build/lib/lm_eval/tasks/glue/sst2/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/sst2 2024-07-01T14:47:40,901 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/rte 2024-07-01T14:47:40,902 copying build/lib/lm_eval/tasks/glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/rte 2024-07-01T14:47:40,904 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/wnli 2024-07-01T14:47:40,905 copying build/lib/lm_eval/tasks/glue/wnli/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/wnli 2024-07-01T14:47:40,907 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qqp 2024-07-01T14:47:40,908 copying build/lib/lm_eval/tasks/glue/qqp/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qqp 2024-07-01T14:47:40,911 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli 2024-07-01T14:47:40,911 copying build/lib/lm_eval/tasks/glue/mnli/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli 2024-07-01T14:47:40,913 copying build/lib/lm_eval/tasks/glue/mnli/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli 2024-07-01T14:47:40,915 copying build/lib/lm_eval/tasks/glue/mnli/mismatch.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli 2024-07-01T14:47:40,917 copying build/lib/lm_eval/tasks/glue/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue 2024-07-01T14:47:40,919 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mrpc 2024-07-01T14:47:40,920 copying build/lib/lm_eval/tasks/glue/mrpc/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mrpc 2024-07-01T14:47:40,923 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qnli 2024-07-01T14:47:40,923 copying build/lib/lm_eval/tasks/glue/qnli/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qnli 2024-07-01T14:47:40,926 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/cola 2024-07-01T14:47:40,927 copying build/lib/lm_eval/tasks/glue/cola/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/cola 2024-07-01T14:47:40,929 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi 2024-07-01T14:47:40,931 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,932 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,934 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,936 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,938 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,939 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,941 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,943 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,945 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,946 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,948 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,950 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,951 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,953 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,955 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,956 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,958 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,960 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,962 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,964 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,965 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,967 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,969 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,971 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,972 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,974 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,976 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,977 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,979 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,981 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,982 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,984 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,986 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,988 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,989 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,991 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,993 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,995 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,997 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:40,999 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,001 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,002 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,004 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,006 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,008 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,010 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,011 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,013 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,014 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,016 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,018 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,019 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,021 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,023 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,024 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,026 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,028 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,030 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,032 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,034 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,036 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,039 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,042 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,046 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,050 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,053 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,055 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2024-07-01T14:47:41,058 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,059 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,060 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,062 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,064 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,065 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,067 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,069 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,071 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,072 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,074 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,076 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,077 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,079 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,081 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,083 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,084 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,086 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,088 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,090 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,092 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,094 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,095 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,097 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,099 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,100 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,102 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,104 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,105 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,107 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,108 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,110 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,111 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,113 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2024-07-01T14:47:41,115 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,116 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,118 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,120 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,122 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,124 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,125 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,127 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,129 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,130 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,132 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,134 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,136 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,137 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,139 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,141 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,143 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,145 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,146 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,148 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,149 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,151 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,153 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,154 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,156 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,158 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,159 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,161 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,163 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,165 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,166 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,168 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,170 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,171 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,174 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,175 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,177 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2024-07-01T14:47:41,180 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,180 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,182 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,184 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,186 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,187 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/_arc_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,189 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,191 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,192 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,194 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,195 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,197 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,199 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,201 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,202 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,204 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,206 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,207 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,209 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,211 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,212 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,214 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,216 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,218 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,220 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,221 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,223 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,224 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,226 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,228 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,229 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,231 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,232 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,234 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,236 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2024-07-01T14:47:41,238 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi 2024-07-01T14:47:41,239 copying build/lib/lm_eval/tasks/babi/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi 2024-07-01T14:47:41,241 copying build/lib/lm_eval/tasks/babi/babi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi 2024-07-01T14:47:41,243 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag 2024-07-01T14:47:41,244 copying build/lib/lm_eval/tasks/swag/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag 2024-07-01T14:47:41,247 copying build/lib/lm_eval/tasks/swag/swag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag 2024-07-01T14:47:41,249 copying build/lib/lm_eval/tasks/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks 2024-07-01T14:47:41,252 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2024-07-01T14:47:41,253 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2024-07-01T14:47:41,255 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2024-07-01T14:47:41,256 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2024-07-01T14:47:41,258 copying build/lib/lm_eval/tasks/truthfulqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2024-07-01T14:47:41,260 copying build/lib/lm_eval/tasks/truthfulqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2024-07-01T14:47:41,263 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue 2024-07-01T14:47:41,264 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,265 copying build/lib/lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,267 copying build/lib/lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,269 copying build/lib/lm_eval/tasks/code_x_glue/code-text/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,271 copying build/lib/lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,272 copying build/lib/lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,274 copying build/lib/lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,276 copying build/lib/lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,278 copying build/lib/lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2024-07-01T14:47:41,280 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,281 copying build/lib/lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,283 copying build/lib/lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,285 copying build/lib/lm_eval/tasks/translation/wmt16_de-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,286 copying build/lib/lm_eval/tasks/translation/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,288 copying build/lib/lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,290 copying build/lib/lm_eval/tasks/translation/wmt_common_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,292 copying build/lib/lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,293 copying build/lib/lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,295 copying build/lib/lm_eval/tasks/translation/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,297 copying build/lib/lm_eval/tasks/translation/wmt16_en-de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,298 copying build/lib/lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2024-07-01T14:47:41,301 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2024-07-01T14:47:41,301 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2024-07-01T14:47:41,303 copying build/lib/lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2024-07-01T14:47:41,305 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2024-07-01T14:47:41,307 copying build/lib/lm_eval/tasks/qa4mre/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2024-07-01T14:47:41,309 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2024-07-01T14:47:41,311 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/copal_id 2024-07-01T14:47:41,311 copying build/lib/lm_eval/tasks/copal_id/standard.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/copal_id 2024-07-01T14:47:41,314 copying build/lib/lm_eval/tasks/copal_id/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/copal_id 2024-07-01T14:47:41,315 copying build/lib/lm_eval/tasks/copal_id/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/copal_id 2024-07-01T14:47:41,317 copying build/lib/lm_eval/tasks/copal_id/colloquial.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/copal_id 2024-07-01T14:47:41,320 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada 2024-07-01T14:47:41,321 copying build/lib/lm_eval/tasks/lambada/lambada_openai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada 2024-07-01T14:47:41,323 copying build/lib/lm_eval/tasks/lambada/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada 2024-07-01T14:47:41,324 copying build/lib/lm_eval/tasks/lambada/lambada_standard.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada 2024-07-01T14:47:41,327 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals 2024-07-01T14:47:41,328 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,330 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,331 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,333 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,335 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,336 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,338 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,339 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,341 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,343 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,344 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,346 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,348 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,350 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,351 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,353 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,355 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,357 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,358 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,360 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,361 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,363 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,365 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,366 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,368 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,370 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,371 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,373 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,375 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,376 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,378 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,380 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,382 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,383 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,385 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,387 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,389 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,390 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,392 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,394 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,395 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,397 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,398 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,400 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,402 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,404 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,405 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,407 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,408 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,410 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,412 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,414 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2024-07-01T14:47:41,416 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:41,417 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:41,419 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:41,420 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy 2024-07-01T14:47:41,423 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/winogenerated 2024-07-01T14:47:41,424 copying build/lib/lm_eval/tasks/model_written_evals/winogenerated/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/winogenerated 2024-07-01T14:47:41,428 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,429 copying build/lib/lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,431 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,432 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,434 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,435 copying build/lib/lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,437 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,439 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,440 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,442 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,444 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,446 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,447 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,449 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,451 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,452 copying build/lib/lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,454 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,456 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,458 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,459 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,461 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,463 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,464 copying build/lib/lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,466 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,468 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,469 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,471 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,473 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,474 copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,476 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,478 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,479 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,481 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,483 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,485 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,487 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,488 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,490 copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,492 copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,494 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,495 copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,497 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,499 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,500 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,502 copying build/lib/lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,504 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,505 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,507 copying build/lib/lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,508 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,510 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,512 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,514 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,516 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,517 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,519 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,521 copying build/lib/lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,523 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,524 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,526 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,528 copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,530 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,535 copying build/lib/lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,536 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,538 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,539 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,541 copying build/lib/lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,542 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,544 copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,546 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,547 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,549 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,551 copying build/lib/lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,552 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,554 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,555 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,557 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,559 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,560 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,562 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,564 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,566 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,567 copying build/lib/lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,569 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,571 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,572 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,574 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,575 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,577 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,578 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,580 copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,582 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,583 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,585 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,586 copying build/lib/lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,588 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,590 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,591 copying build/lib/lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,593 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,595 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,597 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,599 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,600 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,602 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,604 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,606 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,607 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,609 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,610 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,612 copying build/lib/lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,614 copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,615 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,617 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,619 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,621 copying build/lib/lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,622 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,624 copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,626 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,627 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,629 copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,631 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,633 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,635 copying build/lib/lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,637 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,638 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,640 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,642 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,644 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,645 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,647 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,649 copying build/lib/lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,650 copying build/lib/lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,652 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,654 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,655 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,657 copying build/lib/lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,659 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,660 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,662 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2024-07-01T14:47:41,664 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa 2024-07-01T14:47:41,665 copying build/lib/lm_eval/tasks/siqa/siqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa 2024-07-01T14:47:41,667 copying build/lib/lm_eval/tasks/siqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa 2024-07-01T14:47:41,669 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016 2024-07-01T14:47:41,670 copying build/lib/lm_eval/tasks/wmt2016/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016 2024-07-01T14:47:41,672 copying build/lib/lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016 2024-07-01T14:47:41,674 copying build/lib/lm_eval/tasks/wmt2016/metrics.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016 2024-07-01T14:47:41,676 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop 2024-07-01T14:47:41,677 copying build/lib/lm_eval/tasks/drop/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop 2024-07-01T14:47:41,679 copying build/lib/lm_eval/tasks/drop/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop 2024-07-01T14:47:41,681 copying build/lib/lm_eval/tasks/drop/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop 2024-07-01T14:47:41,683 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual 2024-07-01T14:47:41,684 copying build/lib/lm_eval/tasks/mutual/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual 2024-07-01T14:47:41,686 copying build/lib/lm_eval/tasks/mutual/multual_plus.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual 2024-07-01T14:47:41,687 copying build/lib/lm_eval/tasks/mutual/mutual.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual 2024-07-01T14:47:41,689 copying build/lib/lm_eval/tasks/mutual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual 2024-07-01T14:47:41,691 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_proficiency 2024-07-01T14:47:41,692 copying build/lib/lm_eval/tasks/eus_proficiency/eus_proficiency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_proficiency 2024-07-01T14:47:41,693 copying build/lib/lm_eval/tasks/eus_proficiency/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_proficiency 2024-07-01T14:47:41,696 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu 2024-07-01T14:47:41,698 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,699 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,700 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,702 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,704 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,706 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,707 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,709 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,711 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,713 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,714 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,716 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,718 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,720 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,722 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,723 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,725 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,727 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,728 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,730 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,732 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,734 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,735 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,737 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,739 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,740 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,742 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,744 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,746 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,748 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,749 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,751 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,753 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,755 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,757 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,758 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,760 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,762 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,764 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,765 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,767 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,769 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,770 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,772 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,774 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,775 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,777 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2024-07-01T14:47:41,780 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,781 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,783 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,786 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,789 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,795 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,798 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,800 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,802 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,805 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,807 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,809 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,812 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,814 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,817 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,819 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,821 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,823 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,826 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,828 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,831 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,833 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,836 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,838 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,840 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,843 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,845 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,847 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,849 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,852 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,854 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,856 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,859 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,860 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,863 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,865 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,868 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,870 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,872 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,874 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,876 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,878 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,881 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,883 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,886 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,888 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,890 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2024-07-01T14:47:41,893 copying build/lib/lm_eval/tasks/kmmlu/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu 2024-07-01T14:47:41,895 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,896 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,898 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,900 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,901 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,903 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,905 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,906 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,908 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,910 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,911 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,913 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,915 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,917 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,919 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,921 copying build/lib/lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,922 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,924 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,926 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,927 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,929 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,931 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,933 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,934 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,936 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,938 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,940 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,941 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,943 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,945 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,947 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,949 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,950 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,952 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,954 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,956 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,958 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,960 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,961 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,963 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,965 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,967 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,968 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,970 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,972 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,974 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,975 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2024-07-01T14:47:41,978 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,979 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,981 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,982 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,984 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,986 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,988 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,989 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,991 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,993 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,995 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,997 copying build/lib/lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:41,999 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,000 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,002 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,004 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,006 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,008 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,010 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,011 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,013 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,015 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,016 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,018 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,020 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,021 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,023 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,025 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,026 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,028 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,030 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,031 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,033 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,035 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,036 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,038 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,040 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,042 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,043 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,045 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,047 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,049 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,051 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,053 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,054 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,056 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,058 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2024-07-01T14:47:42,060 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2024-07-01T14:47:42,060 copying build/lib/lm_eval/tasks/gsm8k/gsm8k.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2024-07-01T14:47:42,062 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2024-07-01T14:47:42,064 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2024-07-01T14:47:42,066 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2024-07-01T14:47:42,067 copying build/lib/lm_eval/tasks/gsm8k/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2024-07-01T14:47:42,070 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/swde 2024-07-01T14:47:42,070 copying build/lib/lm_eval/tasks/swde/swde.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/swde 2024-07-01T14:47:42,072 copying build/lib/lm_eval/tasks/swde/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/swde 2024-07-01T14:47:42,074 copying build/lib/lm_eval/tasks/swde/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/swde 2024-07-01T14:47:42,077 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noticia 2024-07-01T14:47:42,078 copying build/lib/lm_eval/tasks/noticia/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/noticia 2024-07-01T14:47:42,079 copying build/lib/lm_eval/tasks/noticia/noticia.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/noticia 2024-07-01T14:47:42,081 copying build/lib/lm_eval/tasks/noticia/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/noticia 2024-07-01T14:47:42,084 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile_10k 2024-07-01T14:47:42,084 copying build/lib/lm_eval/tasks/pile_10k/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile_10k 2024-07-01T14:47:42,086 copying build/lib/lm_eval/tasks/pile_10k/pile_10k.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile_10k 2024-07-01T14:47:42,089 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2024-07-01T14:47:42,090 copying build/lib/lm_eval/tasks/xnli_eu/xnli_common_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2024-07-01T14:47:42,091 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2024-07-01T14:47:42,093 copying build/lib/lm_eval/tasks/xnli_eu/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2024-07-01T14:47:42,095 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu_native.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2024-07-01T14:47:42,097 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2024-07-01T14:47:42,099 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2 2024-07-01T14:47:42,100 copying build/lib/lm_eval/tasks/polemo2/polemo2_in.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2 2024-07-01T14:47:42,102 copying build/lib/lm_eval/tasks/polemo2/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2 2024-07-01T14:47:42,104 copying build/lib/lm_eval/tasks/polemo2/polemo2_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2 2024-07-01T14:47:42,106 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:42,107 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:42,108 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:42,110 copying build/lib/lm_eval/tasks/lambada_multilingual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:42,111 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:42,113 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:42,115 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2024-07-01T14:47:42,117 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco 2024-07-01T14:47:42,118 copying build/lib/lm_eval/tasks/mc_taco/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco 2024-07-01T14:47:42,119 copying build/lib/lm_eval/tasks/mc_taco/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco 2024-07-01T14:47:42,122 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,122 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,124 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,126 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,128 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyArc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,130 copying build/lib/lm_eval/tasks/tinyBenchmarks/agg_functions.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,132 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,134 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,135 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,137 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,139 copying build/lib/lm_eval/tasks/tinyBenchmarks/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,141 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,143 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,144 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_winogrande.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2024-07-01T14:47:42,147 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,148 copying build/lib/lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,150 copying build/lib/lm_eval/tasks/kobest/kobest_wic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,152 copying build/lib/lm_eval/tasks/kobest/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,153 copying build/lib/lm_eval/tasks/kobest/kobest_boolq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,155 copying build/lib/lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,156 copying build/lib/lm_eval/tasks/kobest/kobest_copa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,158 copying build/lib/lm_eval/tasks/kobest/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2024-07-01T14:47:42,160 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,161 copying build/lib/lm_eval/tasks/xnli/xnli_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,163 copying build/lib/lm_eval/tasks/xnli/xnli_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,165 copying build/lib/lm_eval/tasks/xnli/xnli_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,166 copying build/lib/lm_eval/tasks/xnli/xnli_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,168 copying build/lib/lm_eval/tasks/xnli/xnli_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,170 copying build/lib/lm_eval/tasks/xnli/xnli_common_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,171 copying build/lib/lm_eval/tasks/xnli/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,174 copying build/lib/lm_eval/tasks/xnli/xnli_ur.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,175 copying build/lib/lm_eval/tasks/xnli/xnli_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,177 copying build/lib/lm_eval/tasks/xnli/xnli_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,179 copying build/lib/lm_eval/tasks/xnli/xnli_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,181 copying build/lib/lm_eval/tasks/xnli/xnli_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,182 copying build/lib/lm_eval/tasks/xnli/xnli_bg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,184 copying build/lib/lm_eval/tasks/xnli/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,187 copying build/lib/lm_eval/tasks/xnli/xnli_tr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,189 copying build/lib/lm_eval/tasks/xnli/xnli_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,191 copying build/lib/lm_eval/tasks/xnli/xnli_el.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,192 copying build/lib/lm_eval/tasks/xnli/xnli_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2024-07-01T14:47:42,195 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue 2024-07-01T14:47:42,196 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte 2024-07-01T14:47:42,197 copying build/lib/lm_eval/tasks/super_glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte 2024-07-01T14:47:42,199 copying build/lib/lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte 2024-07-01T14:47:42,202 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record 2024-07-01T14:47:42,203 copying build/lib/lm_eval/tasks/super_glue/record/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record 2024-07-01T14:47:42,205 copying build/lib/lm_eval/tasks/super_glue/record/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record 2024-07-01T14:47:42,207 copying build/lib/lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record 2024-07-01T14:47:42,208 copying build/lib/lm_eval/tasks/super_glue/record/util.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record 2024-07-01T14:47:42,211 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic 2024-07-01T14:47:42,211 copying build/lib/lm_eval/tasks/super_glue/wic/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic 2024-07-01T14:47:42,213 copying build/lib/lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic 2024-07-01T14:47:42,215 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:42,216 copying build/lib/lm_eval/tasks/super_glue/copa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:42,218 copying build/lib/lm_eval/tasks/super_glue/copa/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:42,219 copying build/lib/lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa 2024-07-01T14:47:42,221 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:42,222 copying build/lib/lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:42,224 copying build/lib/lm_eval/tasks/super_glue/wsc/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:42,226 copying build/lib/lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:42,227 copying build/lib/lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc 2024-07-01T14:47:42,229 copying build/lib/lm_eval/tasks/super_glue/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue 2024-07-01T14:47:42,231 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:42,232 copying build/lib/lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:42,234 copying build/lib/lm_eval/tasks/super_glue/multirc/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:42,236 copying build/lib/lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc 2024-07-01T14:47:42,238 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:42,239 copying build/lib/lm_eval/tasks/super_glue/cb/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:42,240 copying build/lib/lm_eval/tasks/super_glue/cb/aggregate.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:42,242 copying build/lib/lm_eval/tasks/super_glue/cb/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:42,244 copying build/lib/lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb 2024-07-01T14:47:42,246 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:42,247 copying build/lib/lm_eval/tasks/super_glue/boolq/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:42,249 copying build/lib/lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:42,251 copying build/lib/lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq 2024-07-01T14:47:42,253 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,254 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,255 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,257 copying build/lib/lm_eval/tasks/minerva_math/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,260 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,261 copying build/lib/lm_eval/tasks/minerva_math/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,263 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,265 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,267 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,268 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2024-07-01T14:47:42,271 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,271 copying build/lib/lm_eval/tasks/paws-x/paws_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,273 copying build/lib/lm_eval/tasks/paws-x/pawsx_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,275 copying build/lib/lm_eval/tasks/paws-x/paws_ja.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,277 copying build/lib/lm_eval/tasks/paws-x/paws_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,278 copying build/lib/lm_eval/tasks/paws-x/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,280 copying build/lib/lm_eval/tasks/paws-x/_generate_config.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,282 copying build/lib/lm_eval/tasks/paws-x/paws_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,283 copying build/lib/lm_eval/tasks/paws-x/paws_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,285 copying build/lib/lm_eval/tasks/paws-x/paws_ko.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,287 copying build/lib/lm_eval/tasks/paws-x/paws_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2024-07-01T14:47:42,289 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa 2024-07-01T14:47:42,290 copying build/lib/lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa 2024-07-01T14:47:42,291 copying build/lib/lm_eval/tasks/pubmedqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa 2024-07-01T14:47:42,293 copying build/lib/lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa 2024-07-01T14:47:42,295 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,296 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,298 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,300 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,302 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,304 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,306 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,308 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,310 copying build/lib/lm_eval/tasks/arc_mt/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,312 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,314 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,316 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,317 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,319 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2024-07-01T14:47:42,321 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2024-07-01T14:47:42,322 copying build/lib/lm_eval/tasks/ifeval/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2024-07-01T14:47:42,324 copying build/lib/lm_eval/tasks/ifeval/instructions.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2024-07-01T14:47:42,327 copying build/lib/lm_eval/tasks/ifeval/ifeval.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2024-07-01T14:47:42,328 copying build/lib/lm_eval/tasks/ifeval/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2024-07-01T14:47:42,330 copying build/lib/lm_eval/tasks/ifeval/instructions_util.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2024-07-01T14:47:42,333 copying build/lib/lm_eval/tasks/ifeval/instructions_registry.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2024-07-01T14:47:42,336 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,338 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,339 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,341 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,343 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_social_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,345 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,346 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_arabic_language.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,348 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,350 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,351 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,353 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,355 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,357 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,358 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,360 copying build/lib/lm_eval/tasks/arabicmmlu/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,362 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_civics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,364 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,365 copying build/lib/lm_eval/tasks/arabicmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,367 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,369 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,370 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,372 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_prof_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,374 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,376 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_univ_political_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,378 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,379 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,381 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,382 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,384 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,386 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_arabic_language.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,388 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,389 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,391 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,393 copying build/lib/lm_eval/tasks/arabicmmlu/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,395 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_univ_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,396 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,398 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_univ_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,400 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_univ_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,402 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_arabic_language.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,403 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_univ_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,406 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_social_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,407 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,409 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_primary_natural_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,410 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_middle_natural_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,412 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_high_civics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,413 copying build/lib/lm_eval/tasks/arabicmmlu/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2024-07-01T14:47:42,415 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli 2024-07-01T14:47:42,416 copying build/lib/lm_eval/tasks/anli/anli_r1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli 2024-07-01T14:47:42,418 copying build/lib/lm_eval/tasks/anli/anli_r2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli 2024-07-01T14:47:42,420 copying build/lib/lm_eval/tasks/anli/anli_r3.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli 2024-07-01T14:47:42,421 copying build/lib/lm_eval/tasks/anli/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli 2024-07-01T14:47:42,424 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2 2024-07-01T14:47:42,425 copying build/lib/lm_eval/tasks/logiqa2/logieval.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2 2024-07-01T14:47:42,426 copying build/lib/lm_eval/tasks/logiqa2/logiqa2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2 2024-07-01T14:47:42,428 copying build/lib/lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2 2024-07-01T14:47:42,433 copying build/lib/lm_eval/tasks/logiqa2/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2 2024-07-01T14:47:42,436 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,437 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,439 copying build/lib/lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,441 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,443 copying build/lib/lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,445 copying build/lib/lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,447 copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,449 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,451 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,453 copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,455 copying build/lib/lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,457 copying build/lib/lm_eval/tasks/blimp/drop_argument.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,459 copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,461 copying build/lib/lm_eval/tasks/blimp/inchoative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,463 copying build/lib/lm_eval/tasks/blimp/adjunct_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,465 copying build/lib/lm_eval/tasks/blimp/causative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,466 copying build/lib/lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,468 copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,469 copying build/lib/lm_eval/tasks/blimp/only_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,471 copying build/lib/lm_eval/tasks/blimp/intransitive.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,473 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,474 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,476 copying build/lib/lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,478 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,479 copying build/lib/lm_eval/tasks/blimp/wh_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,481 copying build/lib/lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,483 copying build/lib/lm_eval/tasks/blimp/passive_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,484 copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,486 copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,487 copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,489 copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,491 copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,493 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,494 copying build/lib/lm_eval/tasks/blimp/passive_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,496 copying build/lib/lm_eval/tasks/blimp/npi_present_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,498 copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,500 copying build/lib/lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,502 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,504 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,506 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,507 copying build/lib/lm_eval/tasks/blimp/npi_present_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,509 copying build/lib/lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,511 copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,512 copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,514 copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,516 copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,517 copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,519 copying build/lib/lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,521 copying build/lib/lm_eval/tasks/blimp/complex_NP_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,522 copying build/lib/lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,524 copying build/lib/lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,526 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,528 copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,530 copying build/lib/lm_eval/tasks/blimp/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,532 copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,533 copying build/lib/lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,535 copying build/lib/lm_eval/tasks/blimp/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,537 copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,539 copying build/lib/lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,540 copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,542 copying build/lib/lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,544 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,545 copying build/lib/lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,547 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,549 copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,551 copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,553 copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,554 copying build/lib/lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,556 copying build/lib/lm_eval/tasks/blimp/generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,558 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,559 copying build/lib/lm_eval/tasks/blimp/transitive.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2024-07-01T14:47:42,561 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks 2024-07-01T14:47:42,562 copying build/lib/lm_eval/tasks/benchmarks/pythia.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks 2024-07-01T14:47:42,564 copying build/lib/lm_eval/tasks/benchmarks/t0_eval.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks 2024-07-01T14:47:42,566 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa 2024-07-01T14:47:42,567 copying build/lib/lm_eval/tasks/benchmarks/multimedqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa 2024-07-01T14:47:42,569 copying build/lib/lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa 2024-07-01T14:47:42,571 copying build/lib/lm_eval/tasks/benchmarks/minerva_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks 2024-07-01T14:47:42,572 copying build/lib/lm_eval/tasks/benchmarks/openllm.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks 2024-07-01T14:47:42,574 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:42,575 copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:42,577 copying build/lib/lm_eval/tasks/benchmarks/flan/_held_in_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:42,579 copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan 2024-07-01T14:47:42,582 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:42,583 copying build/lib/lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:42,585 copying build/lib/lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:42,586 copying build/lib/lm_eval/tasks/lambada_cloze/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze 2024-07-01T14:47:42,589 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa 2024-07-01T14:47:42,590 copying build/lib/lm_eval/tasks/medqa/medqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa 2024-07-01T14:47:42,591 copying build/lib/lm_eval/tasks/medqa/preprocess_medqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa 2024-07-01T14:47:42,594 copying build/lib/lm_eval/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval 2024-07-01T14:47:42,596 creating build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,597 copying build/lib/lm_eval/api/instance.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,599 copying build/lib/lm_eval/api/samplers.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,601 copying build/lib/lm_eval/api/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,604 copying build/lib/lm_eval/api/registry.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,607 copying build/lib/lm_eval/api/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,608 copying build/lib/lm_eval/api/filter.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,611 copying build/lib/lm_eval/api/model.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,613 copying build/lib/lm_eval/api/metrics.py -> build/bdist.linux-armv7l/wheel/lm_eval/api 2024-07-01T14:47:42,616 creating build/bdist.linux-armv7l/wheel/lm_eval/caching 2024-07-01T14:47:42,617 copying build/lib/lm_eval/caching/cache.py -> build/bdist.linux-armv7l/wheel/lm_eval/caching 2024-07-01T14:47:42,619 creating build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,620 copying build/lib/lm_eval/models/gguf.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,623 copying build/lib/lm_eval/models/neuralmagic.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,625 copying build/lib/lm_eval/models/anthropic_llms.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,627 copying build/lib/lm_eval/models/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,630 copying build/lib/lm_eval/models/vllm_causallms.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,632 copying build/lib/lm_eval/models/dummy.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,634 copying build/lib/lm_eval/models/textsynth.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,636 copying build/lib/lm_eval/models/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,638 copying build/lib/lm_eval/models/nemo_lm.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,640 copying build/lib/lm_eval/models/optimum_lm.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,642 copying build/lib/lm_eval/models/openai_completions.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,645 copying build/lib/lm_eval/models/neuron_optimum.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,647 copying build/lib/lm_eval/models/mamba_lm.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,650 copying build/lib/lm_eval/models/huggingface.py -> build/bdist.linux-armv7l/wheel/lm_eval/models 2024-07-01T14:47:42,654 copying build/lib/lm_eval/__main__.py -> build/bdist.linux-armv7l/wheel/lm_eval 2024-07-01T14:47:42,657 creating build/bdist.linux-armv7l/wheel/lm_eval/filters 2024-07-01T14:47:42,658 copying build/lib/lm_eval/filters/extraction.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters 2024-07-01T14:47:42,660 copying build/lib/lm_eval/filters/transformation.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters 2024-07-01T14:47:42,663 copying build/lib/lm_eval/filters/decontamination.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters 2024-07-01T14:47:42,664 copying build/lib/lm_eval/filters/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters 2024-07-01T14:47:42,666 copying build/lib/lm_eval/filters/selection.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters 2024-07-01T14:47:42,669 creating build/bdist.linux-armv7l/wheel/lm_eval/loggers 2024-07-01T14:47:42,670 copying build/lib/lm_eval/loggers/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/loggers 2024-07-01T14:47:42,672 copying build/lib/lm_eval/loggers/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/loggers 2024-07-01T14:47:42,675 copying build/lib/lm_eval/loggers/wandb_logger.py -> build/bdist.linux-armv7l/wheel/lm_eval/loggers 2024-07-01T14:47:42,678 copying build/lib/lm_eval/loggers/evaluation_tracker.py -> build/bdist.linux-armv7l/wheel/lm_eval/loggers 2024-07-01T14:47:42,681 running install_egg_info 2024-07-01T14:47:42,685 Copying lm_eval.egg-info to build/bdist.linux-armv7l/wheel/lm_eval-0.4.3-py3.11.egg-info 2024-07-01T14:47:42,701 running install_scripts 2024-07-01T14:47:42,740 creating build/bdist.linux-armv7l/wheel/lm_eval-0.4.3.dist-info/WHEEL 2024-07-01T14:47:42,743 creating '/tmp/pip-wheel-3mimzwor/.tmp-_8c_9mdm/lm_eval-0.4.3-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2024-07-01T14:47:42,745 adding 'lm_eval/__init__.py' 2024-07-01T14:47:42,747 adding 'lm_eval/__main__.py' 2024-07-01T14:47:42,751 adding 'lm_eval/evaluator.py' 2024-07-01T14:47:42,753 adding 'lm_eval/evaluator_utils.py' 2024-07-01T14:47:42,756 adding 'lm_eval/utils.py' 2024-07-01T14:47:42,757 adding 'lm_eval/api/__init__.py' 2024-07-01T14:47:42,758 adding 'lm_eval/api/filter.py' 2024-07-01T14:47:42,760 adding 'lm_eval/api/instance.py' 2024-07-01T14:47:42,762 adding 'lm_eval/api/metrics.py' 2024-07-01T14:47:42,764 adding 'lm_eval/api/model.py' 2024-07-01T14:47:42,766 adding 'lm_eval/api/registry.py' 2024-07-01T14:47:42,767 adding 'lm_eval/api/samplers.py' 2024-07-01T14:47:42,774 adding 'lm_eval/api/task.py' 2024-07-01T14:47:42,777 adding 'lm_eval/caching/cache.py' 2024-07-01T14:47:42,778 adding 'lm_eval/decontamination/__init__.py' 2024-07-01T14:47:42,780 adding 'lm_eval/decontamination/archiver.py' 2024-07-01T14:47:42,781 adding 'lm_eval/decontamination/decontaminate.py' 2024-07-01T14:47:42,783 adding 'lm_eval/decontamination/janitor.py' 2024-07-01T14:47:42,785 adding 'lm_eval/filters/__init__.py' 2024-07-01T14:47:42,786 adding 'lm_eval/filters/decontamination.py' 2024-07-01T14:47:42,788 adding 'lm_eval/filters/extraction.py' 2024-07-01T14:47:42,789 adding 'lm_eval/filters/selection.py' 2024-07-01T14:47:42,790 adding 'lm_eval/filters/transformation.py' 2024-07-01T14:47:42,792 adding 'lm_eval/loggers/__init__.py' 2024-07-01T14:47:42,794 adding 'lm_eval/loggers/evaluation_tracker.py' 2024-07-01T14:47:42,796 adding 'lm_eval/loggers/utils.py' 2024-07-01T14:47:42,798 adding 'lm_eval/loggers/wandb_logger.py' 2024-07-01T14:47:42,800 adding 'lm_eval/models/__init__.py' 2024-07-01T14:47:42,802 adding 'lm_eval/models/anthropic_llms.py' 2024-07-01T14:47:42,803 adding 'lm_eval/models/dummy.py' 2024-07-01T14:47:42,804 adding 'lm_eval/models/gguf.py' 2024-07-01T14:47:42,811 adding 'lm_eval/models/huggingface.py' 2024-07-01T14:47:42,812 adding 'lm_eval/models/mamba_lm.py' 2024-07-01T14:47:42,815 adding 'lm_eval/models/nemo_lm.py' 2024-07-01T14:47:42,817 adding 'lm_eval/models/neuralmagic.py' 2024-07-01T14:47:42,821 adding 'lm_eval/models/neuron_optimum.py' 2024-07-01T14:47:42,823 adding 'lm_eval/models/openai_completions.py' 2024-07-01T14:47:42,825 adding 'lm_eval/models/optimum_lm.py' 2024-07-01T14:47:42,826 adding 'lm_eval/models/textsynth.py' 2024-07-01T14:47:42,830 adding 'lm_eval/models/utils.py' 2024-07-01T14:47:42,833 adding 'lm_eval/models/vllm_causallms.py' 2024-07-01T14:47:42,835 adding 'lm_eval/prompts/__init__.py' 2024-07-01T14:47:42,839 adding 'lm_eval/tasks/README.md' 2024-07-01T14:47:42,841 adding 'lm_eval/tasks/__init__.py' 2024-07-01T14:47:42,844 adding 'lm_eval/tasks/aclue/README.md' 2024-07-01T14:47:42,845 adding 'lm_eval/tasks/aclue/_default_template_yaml' 2024-07-01T14:47:42,846 adding 'lm_eval/tasks/aclue/_generate_configs.py' 2024-07-01T14:47:42,848 adding 'lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml' 2024-07-01T14:47:42,849 adding 'lm_eval/tasks/aclue/aclue_ancient_literature.yaml' 2024-07-01T14:47:42,850 adding 'lm_eval/tasks/aclue/aclue_ancient_medical.yaml' 2024-07-01T14:47:42,851 adding 'lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml' 2024-07-01T14:47:42,852 adding 'lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml' 2024-07-01T14:47:42,854 adding 'lm_eval/tasks/aclue/aclue_couplet_prediction.yaml' 2024-07-01T14:47:42,855 adding 'lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml' 2024-07-01T14:47:42,856 adding 'lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml' 2024-07-01T14:47:42,857 adding 'lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml' 2024-07-01T14:47:42,858 adding 'lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml' 2024-07-01T14:47:42,859 adding 'lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml' 2024-07-01T14:47:42,861 adding 'lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml' 2024-07-01T14:47:42,862 adding 'lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml' 2024-07-01T14:47:42,863 adding 'lm_eval/tasks/aclue/aclue_reading_comprehension.yaml' 2024-07-01T14:47:42,864 adding 'lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml' 2024-07-01T14:47:42,866 adding 'lm_eval/tasks/aexams/README.md' 2024-07-01T14:47:42,867 adding 'lm_eval/tasks/aexams/_default_template_yaml' 2024-07-01T14:47:42,869 adding 'lm_eval/tasks/aexams/aexams_Biology.yaml' 2024-07-01T14:47:42,870 adding 'lm_eval/tasks/aexams/aexams_IslamicStudies.yaml' 2024-07-01T14:47:42,871 adding 'lm_eval/tasks/aexams/aexams_Physics.yaml' 2024-07-01T14:47:42,872 adding 'lm_eval/tasks/aexams/aexams_Science.yaml' 2024-07-01T14:47:42,873 adding 'lm_eval/tasks/aexams/aexams_Social.yaml' 2024-07-01T14:47:42,875 adding 'lm_eval/tasks/agieval/README.md' 2024-07-01T14:47:42,876 adding 'lm_eval/tasks/agieval/aqua-rat.yaml' 2024-07-01T14:47:42,877 adding 'lm_eval/tasks/agieval/gaokao-biology.yaml' 2024-07-01T14:47:42,879 adding 'lm_eval/tasks/agieval/gaokao-chemistry.yaml' 2024-07-01T14:47:42,880 adding 'lm_eval/tasks/agieval/gaokao-chinese.yaml' 2024-07-01T14:47:42,881 adding 'lm_eval/tasks/agieval/gaokao-english.yaml' 2024-07-01T14:47:42,882 adding 'lm_eval/tasks/agieval/gaokao-geography.yaml' 2024-07-01T14:47:42,883 adding 'lm_eval/tasks/agieval/gaokao-history.yaml' 2024-07-01T14:47:42,884 adding 'lm_eval/tasks/agieval/gaokao-mathcloze.yaml' 2024-07-01T14:47:42,885 adding 'lm_eval/tasks/agieval/gaokao-mathqa.yaml' 2024-07-01T14:47:42,887 adding 'lm_eval/tasks/agieval/gaokao-physics.yaml' 2024-07-01T14:47:42,888 adding 'lm_eval/tasks/agieval/jec-qa-ca.yaml' 2024-07-01T14:47:42,889 adding 'lm_eval/tasks/agieval/jec-qa-kd.yaml' 2024-07-01T14:47:42,890 adding 'lm_eval/tasks/agieval/logiqa-en.yaml' 2024-07-01T14:47:42,891 adding 'lm_eval/tasks/agieval/logiqa-zh.yaml' 2024-07-01T14:47:42,892 adding 'lm_eval/tasks/agieval/lsat-ar.yaml' 2024-07-01T14:47:42,893 adding 'lm_eval/tasks/agieval/lsat-lr.yaml' 2024-07-01T14:47:42,894 adding 'lm_eval/tasks/agieval/lsat-rc.yaml' 2024-07-01T14:47:42,895 adding 'lm_eval/tasks/agieval/math.yaml' 2024-07-01T14:47:42,896 adding 'lm_eval/tasks/agieval/sat-en-without-passage.yaml' 2024-07-01T14:47:42,898 adding 'lm_eval/tasks/agieval/sat-en.yaml' 2024-07-01T14:47:42,899 adding 'lm_eval/tasks/agieval/sat-math.yaml' 2024-07-01T14:47:42,900 adding 'lm_eval/tasks/agieval/utils.py' 2024-07-01T14:47:42,903 adding 'lm_eval/tasks/alghafa/copa_ar/README.md' 2024-07-01T14:47:42,904 adding 'lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml' 2024-07-01T14:47:42,906 adding 'lm_eval/tasks/alghafa/piqa_ar/README.md' 2024-07-01T14:47:42,907 adding 'lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml' 2024-07-01T14:47:42,909 adding 'lm_eval/tasks/anli/README.md' 2024-07-01T14:47:42,910 adding 'lm_eval/tasks/anli/anli_r1.yaml' 2024-07-01T14:47:42,911 adding 'lm_eval/tasks/anli/anli_r2.yaml' 2024-07-01T14:47:42,913 adding 'lm_eval/tasks/anli/anli_r3.yaml' 2024-07-01T14:47:42,915 adding 'lm_eval/tasks/arabicmmlu/README.md' 2024-07-01T14:47:42,917 adding 'lm_eval/tasks/arabicmmlu/_default_template_yaml' 2024-07-01T14:47:42,918 adding 'lm_eval/tasks/arabicmmlu/_generate_configs.py' 2024-07-01T14:47:42,919 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu.yaml' 2024-07-01T14:47:42,920 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml' 2024-07-01T14:47:42,922 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml' 2024-07-01T14:47:42,923 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml' 2024-07-01T14:47:42,924 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml' 2024-07-01T14:47:42,925 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_arabic_language.yaml' 2024-07-01T14:47:42,926 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_biology.yaml' 2024-07-01T14:47:42,927 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_civics.yaml' 2024-07-01T14:47:42,928 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_computer_science.yaml' 2024-07-01T14:47:42,930 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_economics.yaml' 2024-07-01T14:47:42,931 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_geography.yaml' 2024-07-01T14:47:42,932 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_history.yaml' 2024-07-01T14:47:42,933 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_islamic_studies.yaml' 2024-07-01T14:47:42,934 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_philosophy.yaml' 2024-07-01T14:47:42,935 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_high_physics.yaml' 2024-07-01T14:47:42,936 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml' 2024-07-01T14:47:42,938 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_arabic_language.yaml' 2024-07-01T14:47:42,939 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_civics.yaml' 2024-07-01T14:47:42,940 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_computer_science.yaml' 2024-07-01T14:47:42,942 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_economics.yaml' 2024-07-01T14:47:42,943 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_general_knowledge.yaml' 2024-07-01T14:47:42,944 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_geography.yaml' 2024-07-01T14:47:42,945 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_history.yaml' 2024-07-01T14:47:42,946 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_islamic_studies.yaml' 2024-07-01T14:47:42,947 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_natural_science.yaml' 2024-07-01T14:47:42,949 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_middle_social_science.yaml' 2024-07-01T14:47:42,950 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_arabic_language.yaml' 2024-07-01T14:47:42,951 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_computer_science.yaml' 2024-07-01T14:47:42,952 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_general_knowledge.yaml' 2024-07-01T14:47:42,954 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_geography.yaml' 2024-07-01T14:47:42,955 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_history.yaml' 2024-07-01T14:47:42,956 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_islamic_studies.yaml' 2024-07-01T14:47:42,957 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_math.yaml' 2024-07-01T14:47:42,958 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_natural_science.yaml' 2024-07-01T14:47:42,959 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_primary_social_science.yaml' 2024-07-01T14:47:42,961 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_prof_law.yaml' 2024-07-01T14:47:42,962 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_univ_accounting.yaml' 2024-07-01T14:47:42,963 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_univ_computer_science.yaml' 2024-07-01T14:47:42,964 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_univ_economics.yaml' 2024-07-01T14:47:42,965 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_univ_management.yaml' 2024-07-01T14:47:42,966 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_univ_political_science.yaml' 2024-07-01T14:47:42,968 adding 'lm_eval/tasks/arabicmmlu/utils.py' 2024-07-01T14:47:42,969 adding 'lm_eval/tasks/arc/README.md' 2024-07-01T14:47:42,970 adding 'lm_eval/tasks/arc/arc_challenge.yaml' 2024-07-01T14:47:42,972 adding 'lm_eval/tasks/arc/arc_easy.yaml' 2024-07-01T14:47:42,974 adding 'lm_eval/tasks/arc_mt/README.md' 2024-07-01T14:47:42,975 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml' 2024-07-01T14:47:42,976 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml' 2024-07-01T14:47:42,977 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml' 2024-07-01T14:47:42,978 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml' 2024-07-01T14:47:42,979 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml' 2024-07-01T14:47:42,980 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml' 2024-07-01T14:47:42,982 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml' 2024-07-01T14:47:42,983 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml' 2024-07-01T14:47:42,984 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml' 2024-07-01T14:47:42,985 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml' 2024-07-01T14:47:42,987 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml' 2024-07-01T14:47:42,988 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml' 2024-07-01T14:47:42,990 adding 'lm_eval/tasks/arithmetic/README.md' 2024-07-01T14:47:42,991 adding 'lm_eval/tasks/arithmetic/arithmetic_1dc.yaml' 2024-07-01T14:47:42,992 adding 'lm_eval/tasks/arithmetic/arithmetic_2da.yaml' 2024-07-01T14:47:42,994 adding 'lm_eval/tasks/arithmetic/arithmetic_2dm.yaml' 2024-07-01T14:47:42,995 adding 'lm_eval/tasks/arithmetic/arithmetic_2ds.yaml' 2024-07-01T14:47:42,996 adding 'lm_eval/tasks/arithmetic/arithmetic_3da.yaml' 2024-07-01T14:47:42,997 adding 'lm_eval/tasks/arithmetic/arithmetic_3ds.yaml' 2024-07-01T14:47:42,998 adding 'lm_eval/tasks/arithmetic/arithmetic_4da.yaml' 2024-07-01T14:47:42,999 adding 'lm_eval/tasks/arithmetic/arithmetic_4ds.yaml' 2024-07-01T14:47:43,000 adding 'lm_eval/tasks/arithmetic/arithmetic_5da.yaml' 2024-07-01T14:47:43,001 adding 'lm_eval/tasks/arithmetic/arithmetic_5ds.yaml' 2024-07-01T14:47:43,003 adding 'lm_eval/tasks/asdiv/README.md' 2024-07-01T14:47:43,004 adding 'lm_eval/tasks/asdiv/default.yaml' 2024-07-01T14:47:43,005 adding 'lm_eval/tasks/babi/README.md' 2024-07-01T14:47:43,007 adding 'lm_eval/tasks/babi/babi.yaml' 2024-07-01T14:47:43,008 adding 'lm_eval/tasks/basqueglue/README.md' 2024-07-01T14:47:43,010 adding 'lm_eval/tasks/basqueglue/bec.yaml' 2024-07-01T14:47:43,011 adding 'lm_eval/tasks/basqueglue/bhtc.yaml' 2024-07-01T14:47:43,012 adding 'lm_eval/tasks/basqueglue/coref.yaml' 2024-07-01T14:47:43,013 adding 'lm_eval/tasks/basqueglue/qnli.yaml' 2024-07-01T14:47:43,014 adding 'lm_eval/tasks/basqueglue/utils.py' 2024-07-01T14:47:43,016 adding 'lm_eval/tasks/basqueglue/vaxx.yaml' 2024-07-01T14:47:43,017 adding 'lm_eval/tasks/basqueglue/wic.yaml' 2024-07-01T14:47:43,018 adding 'lm_eval/tasks/bbh/README.md' 2024-07-01T14:47:43,020 adding 'lm_eval/tasks/bbh/_generate_configs.py' 2024-07-01T14:47:43,022 adding 'lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml' 2024-07-01T14:47:43,023 adding 'lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml' 2024-07-01T14:47:43,025 adding 'lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml' 2024-07-01T14:47:43,026 adding 'lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml' 2024-07-01T14:47:43,027 adding 'lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml' 2024-07-01T14:47:43,029 adding 'lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml' 2024-07-01T14:47:43,030 adding 'lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml' 2024-07-01T14:47:43,032 adding 'lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml' 2024-07-01T14:47:43,033 adding 'lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml' 2024-07-01T14:47:43,035 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml' 2024-07-01T14:47:43,036 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml' 2024-07-01T14:47:43,038 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml' 2024-07-01T14:47:43,040 adding 'lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml' 2024-07-01T14:47:43,041 adding 'lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml' 2024-07-01T14:47:43,042 adding 'lm_eval/tasks/bbh/cot_fewshot/navigate.yaml' 2024-07-01T14:47:43,044 adding 'lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml' 2024-07-01T14:47:43,045 adding 'lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml' 2024-07-01T14:47:43,046 adding 'lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml' 2024-07-01T14:47:43,048 adding 'lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml' 2024-07-01T14:47:43,049 adding 'lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml' 2024-07-01T14:47:43,051 adding 'lm_eval/tasks/bbh/cot_fewshot/snarks.yaml' 2024-07-01T14:47:43,052 adding 'lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml' 2024-07-01T14:47:43,053 adding 'lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml' 2024-07-01T14:47:43,054 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml' 2024-07-01T14:47:43,056 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml' 2024-07-01T14:47:43,057 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml' 2024-07-01T14:47:43,058 adding 'lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml' 2024-07-01T14:47:43,060 adding 'lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml' 2024-07-01T14:47:43,062 adding 'lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml' 2024-07-01T14:47:43,063 adding 'lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml' 2024-07-01T14:47:43,064 adding 'lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml' 2024-07-01T14:47:43,065 adding 'lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml' 2024-07-01T14:47:43,066 adding 'lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml' 2024-07-01T14:47:43,067 adding 'lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml' 2024-07-01T14:47:43,068 adding 'lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml' 2024-07-01T14:47:43,070 adding 'lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml' 2024-07-01T14:47:43,071 adding 'lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml' 2024-07-01T14:47:43,072 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml' 2024-07-01T14:47:43,073 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml' 2024-07-01T14:47:43,074 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml' 2024-07-01T14:47:43,075 adding 'lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml' 2024-07-01T14:47:43,076 adding 'lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml' 2024-07-01T14:47:43,077 adding 'lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml' 2024-07-01T14:47:43,078 adding 'lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml' 2024-07-01T14:47:43,080 adding 'lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml' 2024-07-01T14:47:43,081 adding 'lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml' 2024-07-01T14:47:43,082 adding 'lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml' 2024-07-01T14:47:43,083 adding 'lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml' 2024-07-01T14:47:43,084 adding 'lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml' 2024-07-01T14:47:43,086 adding 'lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml' 2024-07-01T14:47:43,087 adding 'lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml' 2024-07-01T14:47:43,088 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml' 2024-07-01T14:47:43,090 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml' 2024-07-01T14:47:43,091 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml' 2024-07-01T14:47:43,092 adding 'lm_eval/tasks/bbh/cot_zeroshot/utils.py' 2024-07-01T14:47:43,094 adding 'lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml' 2024-07-01T14:47:43,095 adding 'lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml' 2024-07-01T14:47:43,098 adding 'lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml' 2024-07-01T14:47:43,099 adding 'lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml' 2024-07-01T14:47:43,101 adding 'lm_eval/tasks/bbh/fewshot/causal_judgement.yaml' 2024-07-01T14:47:43,102 adding 'lm_eval/tasks/bbh/fewshot/date_understanding.yaml' 2024-07-01T14:47:43,103 adding 'lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml' 2024-07-01T14:47:43,105 adding 'lm_eval/tasks/bbh/fewshot/dyck_languages.yaml' 2024-07-01T14:47:43,106 adding 'lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml' 2024-07-01T14:47:43,108 adding 'lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml' 2024-07-01T14:47:43,109 adding 'lm_eval/tasks/bbh/fewshot/hyperbaton.yaml' 2024-07-01T14:47:43,111 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml' 2024-07-01T14:47:43,112 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml' 2024-07-01T14:47:43,114 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml' 2024-07-01T14:47:43,115 adding 'lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml' 2024-07-01T14:47:43,117 adding 'lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml' 2024-07-01T14:47:43,119 adding 'lm_eval/tasks/bbh/fewshot/navigate.yaml' 2024-07-01T14:47:43,120 adding 'lm_eval/tasks/bbh/fewshot/object_counting.yaml' 2024-07-01T14:47:43,122 adding 'lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml' 2024-07-01T14:47:43,123 adding 'lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml' 2024-07-01T14:47:43,125 adding 'lm_eval/tasks/bbh/fewshot/ruin_names.yaml' 2024-07-01T14:47:43,127 adding 'lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml' 2024-07-01T14:47:43,128 adding 'lm_eval/tasks/bbh/fewshot/snarks.yaml' 2024-07-01T14:47:43,130 adding 'lm_eval/tasks/bbh/fewshot/sports_understanding.yaml' 2024-07-01T14:47:43,131 adding 'lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml' 2024-07-01T14:47:43,133 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml' 2024-07-01T14:47:43,135 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml' 2024-07-01T14:47:43,136 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml' 2024-07-01T14:47:43,138 adding 'lm_eval/tasks/bbh/fewshot/web_of_lies.yaml' 2024-07-01T14:47:43,139 adding 'lm_eval/tasks/bbh/fewshot/word_sorting.yaml' 2024-07-01T14:47:43,142 adding 'lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml' 2024-07-01T14:47:43,143 adding 'lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml' 2024-07-01T14:47:43,145 adding 'lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml' 2024-07-01T14:47:43,146 adding 'lm_eval/tasks/bbh/zeroshot/date_understanding.yaml' 2024-07-01T14:47:43,148 adding 'lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml' 2024-07-01T14:47:43,150 adding 'lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml' 2024-07-01T14:47:43,151 adding 'lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml' 2024-07-01T14:47:43,153 adding 'lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml' 2024-07-01T14:47:43,154 adding 'lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml' 2024-07-01T14:47:43,155 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml' 2024-07-01T14:47:43,157 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml' 2024-07-01T14:47:43,158 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml' 2024-07-01T14:47:43,159 adding 'lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml' 2024-07-01T14:47:43,160 adding 'lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml' 2024-07-01T14:47:43,161 adding 'lm_eval/tasks/bbh/zeroshot/navigate.yaml' 2024-07-01T14:47:43,163 adding 'lm_eval/tasks/bbh/zeroshot/object_counting.yaml' 2024-07-01T14:47:43,164 adding 'lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml' 2024-07-01T14:47:43,165 adding 'lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml' 2024-07-01T14:47:43,166 adding 'lm_eval/tasks/bbh/zeroshot/ruin_names.yaml' 2024-07-01T14:47:43,168 adding 'lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml' 2024-07-01T14:47:43,169 adding 'lm_eval/tasks/bbh/zeroshot/snarks.yaml' 2024-07-01T14:47:43,170 adding 'lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml' 2024-07-01T14:47:43,171 adding 'lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml' 2024-07-01T14:47:43,172 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml' 2024-07-01T14:47:43,174 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml' 2024-07-01T14:47:43,175 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml' 2024-07-01T14:47:43,177 adding 'lm_eval/tasks/bbh/zeroshot/utils.py' 2024-07-01T14:47:43,178 adding 'lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml' 2024-07-01T14:47:43,179 adding 'lm_eval/tasks/bbh/zeroshot/word_sorting.yaml' 2024-07-01T14:47:43,183 adding 'lm_eval/tasks/belebele/README.md' 2024-07-01T14:47:43,184 adding 'lm_eval/tasks/belebele/_default_template_yaml' 2024-07-01T14:47:43,186 adding 'lm_eval/tasks/belebele/_generate_configs.py' 2024-07-01T14:47:43,187 adding 'lm_eval/tasks/belebele/belebele_acm_Arab.yaml' 2024-07-01T14:47:43,188 adding 'lm_eval/tasks/belebele/belebele_afr_Latn.yaml' 2024-07-01T14:47:43,189 adding 'lm_eval/tasks/belebele/belebele_als_Latn.yaml' 2024-07-01T14:47:43,190 adding 'lm_eval/tasks/belebele/belebele_amh_Ethi.yaml' 2024-07-01T14:47:43,191 adding 'lm_eval/tasks/belebele/belebele_apc_Arab.yaml' 2024-07-01T14:47:43,192 adding 'lm_eval/tasks/belebele/belebele_arb_Arab.yaml' 2024-07-01T14:47:43,193 adding 'lm_eval/tasks/belebele/belebele_arb_Latn.yaml' 2024-07-01T14:47:43,194 adding 'lm_eval/tasks/belebele/belebele_ars_Arab.yaml' 2024-07-01T14:47:43,195 adding 'lm_eval/tasks/belebele/belebele_ary_Arab.yaml' 2024-07-01T14:47:43,196 adding 'lm_eval/tasks/belebele/belebele_arz_Arab.yaml' 2024-07-01T14:47:43,197 adding 'lm_eval/tasks/belebele/belebele_asm_Beng.yaml' 2024-07-01T14:47:43,198 adding 'lm_eval/tasks/belebele/belebele_azj_Latn.yaml' 2024-07-01T14:47:43,199 adding 'lm_eval/tasks/belebele/belebele_bam_Latn.yaml' 2024-07-01T14:47:43,200 adding 'lm_eval/tasks/belebele/belebele_ben_Beng.yaml' 2024-07-01T14:47:43,201 adding 'lm_eval/tasks/belebele/belebele_ben_Latn.yaml' 2024-07-01T14:47:43,202 adding 'lm_eval/tasks/belebele/belebele_bod_Tibt.yaml' 2024-07-01T14:47:43,204 adding 'lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml' 2024-07-01T14:47:43,205 adding 'lm_eval/tasks/belebele/belebele_cat_Latn.yaml' 2024-07-01T14:47:43,206 adding 'lm_eval/tasks/belebele/belebele_ceb_Latn.yaml' 2024-07-01T14:47:43,207 adding 'lm_eval/tasks/belebele/belebele_ces_Latn.yaml' 2024-07-01T14:47:43,208 adding 'lm_eval/tasks/belebele/belebele_ckb_Arab.yaml' 2024-07-01T14:47:43,209 adding 'lm_eval/tasks/belebele/belebele_dan_Latn.yaml' 2024-07-01T14:47:43,210 adding 'lm_eval/tasks/belebele/belebele_deu_Latn.yaml' 2024-07-01T14:47:43,211 adding 'lm_eval/tasks/belebele/belebele_ell_Grek.yaml' 2024-07-01T14:47:43,212 adding 'lm_eval/tasks/belebele/belebele_eng_Latn.yaml' 2024-07-01T14:47:43,214 adding 'lm_eval/tasks/belebele/belebele_est_Latn.yaml' 2024-07-01T14:47:43,215 adding 'lm_eval/tasks/belebele/belebele_eus_Latn.yaml' 2024-07-01T14:47:43,216 adding 'lm_eval/tasks/belebele/belebele_fin_Latn.yaml' 2024-07-01T14:47:43,217 adding 'lm_eval/tasks/belebele/belebele_fra_Latn.yaml' 2024-07-01T14:47:43,218 adding 'lm_eval/tasks/belebele/belebele_fuv_Latn.yaml' 2024-07-01T14:47:43,219 adding 'lm_eval/tasks/belebele/belebele_gaz_Latn.yaml' 2024-07-01T14:47:43,220 adding 'lm_eval/tasks/belebele/belebele_grn_Latn.yaml' 2024-07-01T14:47:43,222 adding 'lm_eval/tasks/belebele/belebele_guj_Gujr.yaml' 2024-07-01T14:47:43,223 adding 'lm_eval/tasks/belebele/belebele_hat_Latn.yaml' 2024-07-01T14:47:43,224 adding 'lm_eval/tasks/belebele/belebele_hau_Latn.yaml' 2024-07-01T14:47:43,225 adding 'lm_eval/tasks/belebele/belebele_heb_Hebr.yaml' 2024-07-01T14:47:43,226 adding 'lm_eval/tasks/belebele/belebele_hin_Deva.yaml' 2024-07-01T14:47:43,227 adding 'lm_eval/tasks/belebele/belebele_hin_Latn.yaml' 2024-07-01T14:47:43,228 adding 'lm_eval/tasks/belebele/belebele_hrv_Latn.yaml' 2024-07-01T14:47:43,230 adding 'lm_eval/tasks/belebele/belebele_hun_Latn.yaml' 2024-07-01T14:47:43,231 adding 'lm_eval/tasks/belebele/belebele_hye_Armn.yaml' 2024-07-01T14:47:43,232 adding 'lm_eval/tasks/belebele/belebele_ibo_Latn.yaml' 2024-07-01T14:47:43,233 adding 'lm_eval/tasks/belebele/belebele_ilo_Latn.yaml' 2024-07-01T14:47:43,234 adding 'lm_eval/tasks/belebele/belebele_ind_Latn.yaml' 2024-07-01T14:47:43,235 adding 'lm_eval/tasks/belebele/belebele_isl_Latn.yaml' 2024-07-01T14:47:43,236 adding 'lm_eval/tasks/belebele/belebele_ita_Latn.yaml' 2024-07-01T14:47:43,238 adding 'lm_eval/tasks/belebele/belebele_jav_Latn.yaml' 2024-07-01T14:47:43,239 adding 'lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml' 2024-07-01T14:47:43,240 adding 'lm_eval/tasks/belebele/belebele_kac_Latn.yaml' 2024-07-01T14:47:43,241 adding 'lm_eval/tasks/belebele/belebele_kan_Knda.yaml' 2024-07-01T14:47:43,243 adding 'lm_eval/tasks/belebele/belebele_kat_Geor.yaml' 2024-07-01T14:47:43,244 adding 'lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml' 2024-07-01T14:47:43,245 adding 'lm_eval/tasks/belebele/belebele_kea_Latn.yaml' 2024-07-01T14:47:43,246 adding 'lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml' 2024-07-01T14:47:43,247 adding 'lm_eval/tasks/belebele/belebele_khm_Khmr.yaml' 2024-07-01T14:47:43,248 adding 'lm_eval/tasks/belebele/belebele_kin_Latn.yaml' 2024-07-01T14:47:43,249 adding 'lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml' 2024-07-01T14:47:43,250 adding 'lm_eval/tasks/belebele/belebele_kor_Hang.yaml' 2024-07-01T14:47:43,251 adding 'lm_eval/tasks/belebele/belebele_lao_Laoo.yaml' 2024-07-01T14:47:43,252 adding 'lm_eval/tasks/belebele/belebele_lin_Latn.yaml' 2024-07-01T14:47:43,253 adding 'lm_eval/tasks/belebele/belebele_lit_Latn.yaml' 2024-07-01T14:47:43,254 adding 'lm_eval/tasks/belebele/belebele_lug_Latn.yaml' 2024-07-01T14:47:43,255 adding 'lm_eval/tasks/belebele/belebele_luo_Latn.yaml' 2024-07-01T14:47:43,256 adding 'lm_eval/tasks/belebele/belebele_lvs_Latn.yaml' 2024-07-01T14:47:43,257 adding 'lm_eval/tasks/belebele/belebele_mal_Mlym.yaml' 2024-07-01T14:47:43,258 adding 'lm_eval/tasks/belebele/belebele_mar_Deva.yaml' 2024-07-01T14:47:43,259 adding 'lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml' 2024-07-01T14:47:43,260 adding 'lm_eval/tasks/belebele/belebele_mlt_Latn.yaml' 2024-07-01T14:47:43,261 adding 'lm_eval/tasks/belebele/belebele_mri_Latn.yaml' 2024-07-01T14:47:43,262 adding 'lm_eval/tasks/belebele/belebele_mya_Mymr.yaml' 2024-07-01T14:47:43,264 adding 'lm_eval/tasks/belebele/belebele_nld_Latn.yaml' 2024-07-01T14:47:43,265 adding 'lm_eval/tasks/belebele/belebele_nob_Latn.yaml' 2024-07-01T14:47:43,266 adding 'lm_eval/tasks/belebele/belebele_npi_Deva.yaml' 2024-07-01T14:47:43,267 adding 'lm_eval/tasks/belebele/belebele_npi_Latn.yaml' 2024-07-01T14:47:43,268 adding 'lm_eval/tasks/belebele/belebele_nso_Latn.yaml' 2024-07-01T14:47:43,269 adding 'lm_eval/tasks/belebele/belebele_nya_Latn.yaml' 2024-07-01T14:47:43,270 adding 'lm_eval/tasks/belebele/belebele_ory_Orya.yaml' 2024-07-01T14:47:43,271 adding 'lm_eval/tasks/belebele/belebele_pan_Guru.yaml' 2024-07-01T14:47:43,273 adding 'lm_eval/tasks/belebele/belebele_pbt_Arab.yaml' 2024-07-01T14:47:43,274 adding 'lm_eval/tasks/belebele/belebele_pes_Arab.yaml' 2024-07-01T14:47:43,275 adding 'lm_eval/tasks/belebele/belebele_plt_Latn.yaml' 2024-07-01T14:47:43,276 adding 'lm_eval/tasks/belebele/belebele_pol_Latn.yaml' 2024-07-01T14:47:43,277 adding 'lm_eval/tasks/belebele/belebele_por_Latn.yaml' 2024-07-01T14:47:43,278 adding 'lm_eval/tasks/belebele/belebele_ron_Latn.yaml' 2024-07-01T14:47:43,279 adding 'lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml' 2024-07-01T14:47:43,280 adding 'lm_eval/tasks/belebele/belebele_shn_Mymr.yaml' 2024-07-01T14:47:43,282 adding 'lm_eval/tasks/belebele/belebele_sin_Latn.yaml' 2024-07-01T14:47:43,283 adding 'lm_eval/tasks/belebele/belebele_sin_Sinh.yaml' 2024-07-01T14:47:43,284 adding 'lm_eval/tasks/belebele/belebele_slk_Latn.yaml' 2024-07-01T14:47:43,285 adding 'lm_eval/tasks/belebele/belebele_slv_Latn.yaml' 2024-07-01T14:47:43,286 adding 'lm_eval/tasks/belebele/belebele_sna_Latn.yaml' 2024-07-01T14:47:43,287 adding 'lm_eval/tasks/belebele/belebele_snd_Arab.yaml' 2024-07-01T14:47:43,288 adding 'lm_eval/tasks/belebele/belebele_som_Latn.yaml' 2024-07-01T14:47:43,290 adding 'lm_eval/tasks/belebele/belebele_sot_Latn.yaml' 2024-07-01T14:47:43,291 adding 'lm_eval/tasks/belebele/belebele_spa_Latn.yaml' 2024-07-01T14:47:43,292 adding 'lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml' 2024-07-01T14:47:43,293 adding 'lm_eval/tasks/belebele/belebele_ssw_Latn.yaml' 2024-07-01T14:47:43,294 adding 'lm_eval/tasks/belebele/belebele_sun_Latn.yaml' 2024-07-01T14:47:43,295 adding 'lm_eval/tasks/belebele/belebele_swe_Latn.yaml' 2024-07-01T14:47:43,297 adding 'lm_eval/tasks/belebele/belebele_swh_Latn.yaml' 2024-07-01T14:47:43,298 adding 'lm_eval/tasks/belebele/belebele_tam_Taml.yaml' 2024-07-01T14:47:43,299 adding 'lm_eval/tasks/belebele/belebele_tel_Telu.yaml' 2024-07-01T14:47:43,300 adding 'lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml' 2024-07-01T14:47:43,301 adding 'lm_eval/tasks/belebele/belebele_tgl_Latn.yaml' 2024-07-01T14:47:43,302 adding 'lm_eval/tasks/belebele/belebele_tha_Thai.yaml' 2024-07-01T14:47:43,303 adding 'lm_eval/tasks/belebele/belebele_tir_Ethi.yaml' 2024-07-01T14:47:43,304 adding 'lm_eval/tasks/belebele/belebele_tsn_Latn.yaml' 2024-07-01T14:47:43,305 adding 'lm_eval/tasks/belebele/belebele_tso_Latn.yaml' 2024-07-01T14:47:43,306 adding 'lm_eval/tasks/belebele/belebele_tur_Latn.yaml' 2024-07-01T14:47:43,307 adding 'lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml' 2024-07-01T14:47:43,308 adding 'lm_eval/tasks/belebele/belebele_urd_Arab.yaml' 2024-07-01T14:47:43,309 adding 'lm_eval/tasks/belebele/belebele_urd_Latn.yaml' 2024-07-01T14:47:43,310 adding 'lm_eval/tasks/belebele/belebele_uzn_Latn.yaml' 2024-07-01T14:47:43,311 adding 'lm_eval/tasks/belebele/belebele_vie_Latn.yaml' 2024-07-01T14:47:43,312 adding 'lm_eval/tasks/belebele/belebele_war_Latn.yaml' 2024-07-01T14:47:43,313 adding 'lm_eval/tasks/belebele/belebele_wol_Latn.yaml' 2024-07-01T14:47:43,314 adding 'lm_eval/tasks/belebele/belebele_xho_Latn.yaml' 2024-07-01T14:47:43,315 adding 'lm_eval/tasks/belebele/belebele_yor_Latn.yaml' 2024-07-01T14:47:43,316 adding 'lm_eval/tasks/belebele/belebele_zho_Hans.yaml' 2024-07-01T14:47:43,317 adding 'lm_eval/tasks/belebele/belebele_zho_Hant.yaml' 2024-07-01T14:47:43,318 adding 'lm_eval/tasks/belebele/belebele_zsm_Latn.yaml' 2024-07-01T14:47:43,320 adding 'lm_eval/tasks/belebele/belebele_zul_Latn.yaml' 2024-07-01T14:47:43,321 adding 'lm_eval/tasks/benchmarks/minerva_math.yaml' 2024-07-01T14:47:43,323 adding 'lm_eval/tasks/benchmarks/openllm.yaml' 2024-07-01T14:47:43,324 adding 'lm_eval/tasks/benchmarks/pythia.yaml' 2024-07-01T14:47:43,325 adding 'lm_eval/tasks/benchmarks/t0_eval.yaml' 2024-07-01T14:47:43,327 adding 'lm_eval/tasks/benchmarks/flan/_held_in_template_yaml' 2024-07-01T14:47:43,329 adding 'lm_eval/tasks/benchmarks/flan/flan_held_in.yaml' 2024-07-01T14:47:43,330 adding 'lm_eval/tasks/benchmarks/flan/flan_held_out.yaml' 2024-07-01T14:47:43,332 adding 'lm_eval/tasks/benchmarks/multimedqa/README.md' 2024-07-01T14:47:43,333 adding 'lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml' 2024-07-01T14:47:43,335 adding 'lm_eval/tasks/bertaqa/README.md' 2024-07-01T14:47:43,336 adding 'lm_eval/tasks/bertaqa/_bertaqa_template' 2024-07-01T14:47:43,337 adding 'lm_eval/tasks/bertaqa/bertaqa_en.yaml' 2024-07-01T14:47:43,338 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml' 2024-07-01T14:47:43,339 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml' 2024-07-01T14:47:43,341 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml' 2024-07-01T14:47:43,342 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml' 2024-07-01T14:47:43,343 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml' 2024-07-01T14:47:43,344 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml' 2024-07-01T14:47:43,345 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml' 2024-07-01T14:47:43,346 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml' 2024-07-01T14:47:43,348 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml' 2024-07-01T14:47:43,349 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml' 2024-07-01T14:47:43,350 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml' 2024-07-01T14:47:43,351 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml' 2024-07-01T14:47:43,352 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml' 2024-07-01T14:47:43,353 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml' 2024-07-01T14:47:43,355 adding 'lm_eval/tasks/bertaqa/bertaqa_eu.yaml' 2024-07-01T14:47:43,357 adding 'lm_eval/tasks/bigbench/README.md' 2024-07-01T14:47:43,359 adding 'lm_eval/tasks/bigbench/generate_tasks.py' 2024-07-01T14:47:43,360 adding 'lm_eval/tasks/bigbench/generate_until_template_yaml' 2024-07-01T14:47:43,361 adding 'lm_eval/tasks/bigbench/multiple_choice_template_a_yaml' 2024-07-01T14:47:43,362 adding 'lm_eval/tasks/bigbench/multiple_choice_template_b_yaml' 2024-07-01T14:47:43,363 adding 'lm_eval/tasks/bigbench/push_bigbench_dataset.py' 2024-07-01T14:47:43,368 adding 'lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml' 2024-07-01T14:47:43,369 adding 'lm_eval/tasks/bigbench/generate_until/anachronisms.yaml' 2024-07-01T14:47:43,370 adding 'lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml' 2024-07-01T14:47:43,371 adding 'lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml' 2024-07-01T14:47:43,372 adding 'lm_eval/tasks/bigbench/generate_until/arithmetic.yaml' 2024-07-01T14:47:43,373 adding 'lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml' 2024-07-01T14:47:43,374 adding 'lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml' 2024-07-01T14:47:43,375 adding 'lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml' 2024-07-01T14:47:43,377 adding 'lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml' 2024-07-01T14:47:43,378 adding 'lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml' 2024-07-01T14:47:43,379 adding 'lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml' 2024-07-01T14:47:43,380 adding 'lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml' 2024-07-01T14:47:43,381 adding 'lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml' 2024-07-01T14:47:43,382 adding 'lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml' 2024-07-01T14:47:43,383 adding 'lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml' 2024-07-01T14:47:43,384 adding 'lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml' 2024-07-01T14:47:43,386 adding 'lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml' 2024-07-01T14:47:43,387 adding 'lm_eval/tasks/bigbench/generate_until/code_line_description.yaml' 2024-07-01T14:47:43,388 adding 'lm_eval/tasks/bigbench/generate_until/codenames.yaml' 2024-07-01T14:47:43,389 adding 'lm_eval/tasks/bigbench/generate_until/color.yaml' 2024-07-01T14:47:43,390 adding 'lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml' 2024-07-01T14:47:43,391 adding 'lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml' 2024-07-01T14:47:43,392 adding 'lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml' 2024-07-01T14:47:43,394 adding 'lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml' 2024-07-01T14:47:43,395 adding 'lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml' 2024-07-01T14:47:43,396 adding 'lm_eval/tasks/bigbench/generate_until/crass_ai.yaml' 2024-07-01T14:47:43,397 adding 'lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml' 2024-07-01T14:47:43,398 adding 'lm_eval/tasks/bigbench/generate_until/cryptonite.yaml' 2024-07-01T14:47:43,399 adding 'lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml' 2024-07-01T14:47:43,401 adding 'lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml' 2024-07-01T14:47:43,402 adding 'lm_eval/tasks/bigbench/generate_until/date_understanding.yaml' 2024-07-01T14:47:43,403 adding 'lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml' 2024-07-01T14:47:43,404 adding 'lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml' 2024-07-01T14:47:43,405 adding 'lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml' 2024-07-01T14:47:43,406 adding 'lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml' 2024-07-01T14:47:43,408 adding 'lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml' 2024-07-01T14:47:43,409 adding 'lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml' 2024-07-01T14:47:43,410 adding 'lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml' 2024-07-01T14:47:43,411 adding 'lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml' 2024-07-01T14:47:43,412 adding 'lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml' 2024-07-01T14:47:43,413 adding 'lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml' 2024-07-01T14:47:43,414 adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml' 2024-07-01T14:47:43,415 adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml' 2024-07-01T14:47:43,416 adding 'lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml' 2024-07-01T14:47:43,417 adding 'lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml' 2024-07-01T14:47:43,418 adding 'lm_eval/tasks/bigbench/generate_until/fact_checker.yaml' 2024-07-01T14:47:43,420 adding 'lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml' 2024-07-01T14:47:43,421 adding 'lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml' 2024-07-01T14:47:43,422 adding 'lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml' 2024-07-01T14:47:43,423 adding 'lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml' 2024-07-01T14:47:43,424 adding 'lm_eval/tasks/bigbench/generate_until/gem.yaml' 2024-07-01T14:47:43,425 adding 'lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml' 2024-07-01T14:47:43,426 adding 'lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml' 2024-07-01T14:47:43,427 adding 'lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml' 2024-07-01T14:47:43,428 adding 'lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml' 2024-07-01T14:47:43,429 adding 'lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml' 2024-07-01T14:47:43,431 adding 'lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml' 2024-07-01T14:47:43,432 adding 'lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml' 2024-07-01T14:47:43,433 adding 'lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml' 2024-07-01T14:47:43,434 adding 'lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml' 2024-07-01T14:47:43,435 adding 'lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml' 2024-07-01T14:47:43,436 adding 'lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml' 2024-07-01T14:47:43,437 adding 'lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml' 2024-07-01T14:47:43,438 adding 'lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml' 2024-07-01T14:47:43,439 adding 'lm_eval/tasks/bigbench/generate_until/implicatures.yaml' 2024-07-01T14:47:43,441 adding 'lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml' 2024-07-01T14:47:43,442 adding 'lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml' 2024-07-01T14:47:43,443 adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml' 2024-07-01T14:47:43,444 adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml' 2024-07-01T14:47:43,445 adding 'lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml' 2024-07-01T14:47:43,446 adding 'lm_eval/tasks/bigbench/generate_until/irony_identification.yaml' 2024-07-01T14:47:43,447 adding 'lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml' 2024-07-01T14:47:43,449 adding 'lm_eval/tasks/bigbench/generate_until/kannada.yaml' 2024-07-01T14:47:43,450 adding 'lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml' 2024-07-01T14:47:43,451 adding 'lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml' 2024-07-01T14:47:43,452 adding 'lm_eval/tasks/bigbench/generate_until/language_games.yaml' 2024-07-01T14:47:43,453 adding 'lm_eval/tasks/bigbench/generate_until/language_identification.yaml' 2024-07-01T14:47:43,454 adding 'lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml' 2024-07-01T14:47:43,455 adding 'lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml' 2024-07-01T14:47:43,457 adding 'lm_eval/tasks/bigbench/generate_until/list_functions.yaml' 2024-07-01T14:47:43,458 adding 'lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml' 2024-07-01T14:47:43,459 adding 'lm_eval/tasks/bigbench/generate_until/logical_args.yaml' 2024-07-01T14:47:43,460 adding 'lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml' 2024-07-01T14:47:43,461 adding 'lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml' 2024-07-01T14:47:43,462 adding 'lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml' 2024-07-01T14:47:43,463 adding 'lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml' 2024-07-01T14:47:43,464 adding 'lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml' 2024-07-01T14:47:43,465 adding 'lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml' 2024-07-01T14:47:43,466 adding 'lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml' 2024-07-01T14:47:43,467 adding 'lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml' 2024-07-01T14:47:43,468 adding 'lm_eval/tasks/bigbench/generate_until/misconceptions.yaml' 2024-07-01T14:47:43,469 adding 'lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml' 2024-07-01T14:47:43,470 adding 'lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml' 2024-07-01T14:47:43,471 adding 'lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml' 2024-07-01T14:47:43,472 adding 'lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml' 2024-07-01T14:47:43,474 adding 'lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml' 2024-07-01T14:47:43,475 adding 'lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml' 2024-07-01T14:47:43,476 adding 'lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml' 2024-07-01T14:47:43,477 adding 'lm_eval/tasks/bigbench/generate_until/multiemo.yaml' 2024-07-01T14:47:43,478 adding 'lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml' 2024-07-01T14:47:43,479 adding 'lm_eval/tasks/bigbench/generate_until/navigate.yaml' 2024-07-01T14:47:43,480 adding 'lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml' 2024-07-01T14:47:43,481 adding 'lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml' 2024-07-01T14:47:43,483 adding 'lm_eval/tasks/bigbench/generate_until/object_counting.yaml' 2024-07-01T14:47:43,484 adding 'lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml' 2024-07-01T14:47:43,485 adding 'lm_eval/tasks/bigbench/generate_until/operators.yaml' 2024-07-01T14:47:43,486 adding 'lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml' 2024-07-01T14:47:43,487 adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml' 2024-07-01T14:47:43,488 adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml' 2024-07-01T14:47:43,489 adding 'lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml' 2024-07-01T14:47:43,491 adding 'lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml' 2024-07-01T14:47:43,492 adding 'lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml' 2024-07-01T14:47:43,493 adding 'lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml' 2024-07-01T14:47:43,494 adding 'lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml' 2024-07-01T14:47:43,495 adding 'lm_eval/tasks/bigbench/generate_until/physics.yaml' 2024-07-01T14:47:43,496 adding 'lm_eval/tasks/bigbench/generate_until/physics_questions.yaml' 2024-07-01T14:47:43,498 adding 'lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml' 2024-07-01T14:47:43,499 adding 'lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml' 2024-07-01T14:47:43,500 adding 'lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml' 2024-07-01T14:47:43,501 adding 'lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml' 2024-07-01T14:47:43,503 adding 'lm_eval/tasks/bigbench/generate_until/question_selection.yaml' 2024-07-01T14:47:43,504 adding 'lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml' 2024-07-01T14:47:43,505 adding 'lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml' 2024-07-01T14:47:43,506 adding 'lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml' 2024-07-01T14:47:43,507 adding 'lm_eval/tasks/bigbench/generate_until/rephrase.yaml' 2024-07-01T14:47:43,508 adding 'lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml' 2024-07-01T14:47:43,510 adding 'lm_eval/tasks/bigbench/generate_until/ruin_names.yaml' 2024-07-01T14:47:43,511 adding 'lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml' 2024-07-01T14:47:43,512 adding 'lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml' 2024-07-01T14:47:43,513 adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml' 2024-07-01T14:47:43,514 adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml' 2024-07-01T14:47:43,515 adding 'lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml' 2024-07-01T14:47:43,516 adding 'lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml' 2024-07-01T14:47:43,518 adding 'lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml' 2024-07-01T14:47:43,519 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml' 2024-07-01T14:47:43,520 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml' 2024-07-01T14:47:43,521 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml' 2024-07-01T14:47:43,522 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml' 2024-07-01T14:47:43,523 adding 'lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml' 2024-07-01T14:47:43,524 adding 'lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml' 2024-07-01T14:47:43,526 adding 'lm_eval/tasks/bigbench/generate_until/snarks.yaml' 2024-07-01T14:47:43,527 adding 'lm_eval/tasks/bigbench/generate_until/social_iqa.yaml' 2024-07-01T14:47:43,528 adding 'lm_eval/tasks/bigbench/generate_until/social_support.yaml' 2024-07-01T14:47:43,529 adding 'lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml' 2024-07-01T14:47:43,530 adding 'lm_eval/tasks/bigbench/generate_until/strange_stories.yaml' 2024-07-01T14:47:43,531 adding 'lm_eval/tasks/bigbench/generate_until/strategyqa.yaml' 2024-07-01T14:47:43,532 adding 'lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml' 2024-07-01T14:47:43,534 adding 'lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml' 2024-07-01T14:47:43,535 adding 'lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml' 2024-07-01T14:47:43,536 adding 'lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml' 2024-07-01T14:47:43,537 adding 'lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml' 2024-07-01T14:47:43,538 adding 'lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml' 2024-07-01T14:47:43,539 adding 'lm_eval/tasks/bigbench/generate_until/tense.yaml' 2024-07-01T14:47:43,540 adding 'lm_eval/tasks/bigbench/generate_until/timedial.yaml' 2024-07-01T14:47:43,541 adding 'lm_eval/tasks/bigbench/generate_until/topical_chat.yaml' 2024-07-01T14:47:43,543 adding 'lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml' 2024-07-01T14:47:43,544 adding 'lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml' 2024-07-01T14:47:43,545 adding 'lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml' 2024-07-01T14:47:43,546 adding 'lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml' 2024-07-01T14:47:43,547 adding 'lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml' 2024-07-01T14:47:43,548 adding 'lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml' 2024-07-01T14:47:43,549 adding 'lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml' 2024-07-01T14:47:43,551 adding 'lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml' 2024-07-01T14:47:43,552 adding 'lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml' 2024-07-01T14:47:43,553 adding 'lm_eval/tasks/bigbench/generate_until/winowhy.yaml' 2024-07-01T14:47:43,554 adding 'lm_eval/tasks/bigbench/generate_until/word_sorting.yaml' 2024-07-01T14:47:43,555 adding 'lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml' 2024-07-01T14:47:43,558 adding 'lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml' 2024-07-01T14:47:43,559 adding 'lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml' 2024-07-01T14:47:43,560 adding 'lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml' 2024-07-01T14:47:43,561 adding 'lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml' 2024-07-01T14:47:43,563 adding 'lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml' 2024-07-01T14:47:43,564 adding 'lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml' 2024-07-01T14:47:43,565 adding 'lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml' 2024-07-01T14:47:43,566 adding 'lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml' 2024-07-01T14:47:43,567 adding 'lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml' 2024-07-01T14:47:43,568 adding 'lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml' 2024-07-01T14:47:43,569 adding 'lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml' 2024-07-01T14:47:43,570 adding 'lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml' 2024-07-01T14:47:43,572 adding 'lm_eval/tasks/bigbench/multiple_choice/color.yaml' 2024-07-01T14:47:43,573 adding 'lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml' 2024-07-01T14:47:43,574 adding 'lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml' 2024-07-01T14:47:43,575 adding 'lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml' 2024-07-01T14:47:43,576 adding 'lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml' 2024-07-01T14:47:43,577 adding 'lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml' 2024-07-01T14:47:43,578 adding 'lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml' 2024-07-01T14:47:43,579 adding 'lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml' 2024-07-01T14:47:43,580 adding 'lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml' 2024-07-01T14:47:43,581 adding 'lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml' 2024-07-01T14:47:43,583 adding 'lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml' 2024-07-01T14:47:43,584 adding 'lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml' 2024-07-01T14:47:43,585 adding 'lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml' 2024-07-01T14:47:43,586 adding 'lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml' 2024-07-01T14:47:43,587 adding 'lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml' 2024-07-01T14:47:43,588 adding 'lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml' 2024-07-01T14:47:43,589 adding 'lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml' 2024-07-01T14:47:43,591 adding 'lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml' 2024-07-01T14:47:43,592 adding 'lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml' 2024-07-01T14:47:43,593 adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml' 2024-07-01T14:47:43,594 adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml' 2024-07-01T14:47:43,595 adding 'lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml' 2024-07-01T14:47:43,596 adding 'lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml' 2024-07-01T14:47:43,598 adding 'lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml' 2024-07-01T14:47:43,599 adding 'lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml' 2024-07-01T14:47:43,600 adding 'lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml' 2024-07-01T14:47:43,601 adding 'lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml' 2024-07-01T14:47:43,602 adding 'lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml' 2024-07-01T14:47:43,603 adding 'lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml' 2024-07-01T14:47:43,604 adding 'lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml' 2024-07-01T14:47:43,605 adding 'lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml' 2024-07-01T14:47:43,606 adding 'lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml' 2024-07-01T14:47:43,607 adding 'lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml' 2024-07-01T14:47:43,608 adding 'lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml' 2024-07-01T14:47:43,610 adding 'lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml' 2024-07-01T14:47:43,611 adding 'lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml' 2024-07-01T14:47:43,612 adding 'lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml' 2024-07-01T14:47:43,613 adding 'lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml' 2024-07-01T14:47:43,614 adding 'lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml' 2024-07-01T14:47:43,615 adding 'lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml' 2024-07-01T14:47:43,616 adding 'lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml' 2024-07-01T14:47:43,618 adding 'lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml' 2024-07-01T14:47:43,619 adding 'lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml' 2024-07-01T14:47:43,620 adding 'lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml' 2024-07-01T14:47:43,621 adding 'lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml' 2024-07-01T14:47:43,622 adding 'lm_eval/tasks/bigbench/multiple_choice/kannada.yaml' 2024-07-01T14:47:43,623 adding 'lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml' 2024-07-01T14:47:43,624 adding 'lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml' 2024-07-01T14:47:43,626 adding 'lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml' 2024-07-01T14:47:43,627 adding 'lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml' 2024-07-01T14:47:43,628 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml' 2024-07-01T14:47:43,629 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml' 2024-07-01T14:47:43,630 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml' 2024-07-01T14:47:43,631 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml' 2024-07-01T14:47:43,632 adding 'lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml' 2024-07-01T14:47:43,633 adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml' 2024-07-01T14:47:43,635 adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml' 2024-07-01T14:47:43,636 adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml' 2024-07-01T14:47:43,637 adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml' 2024-07-01T14:47:43,638 adding 'lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml' 2024-07-01T14:47:43,639 adding 'lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml' 2024-07-01T14:47:43,640 adding 'lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml' 2024-07-01T14:47:43,642 adding 'lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml' 2024-07-01T14:47:43,643 adding 'lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml' 2024-07-01T14:47:43,644 adding 'lm_eval/tasks/bigbench/multiple_choice/navigate.yaml' 2024-07-01T14:47:43,645 adding 'lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml' 2024-07-01T14:47:43,647 adding 'lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml' 2024-07-01T14:47:43,648 adding 'lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml' 2024-07-01T14:47:43,649 adding 'lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml' 2024-07-01T14:47:43,650 adding 'lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml' 2024-07-01T14:47:43,651 adding 'lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml' 2024-07-01T14:47:43,652 adding 'lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml' 2024-07-01T14:47:43,654 adding 'lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml' 2024-07-01T14:47:43,655 adding 'lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml' 2024-07-01T14:47:43,656 adding 'lm_eval/tasks/bigbench/multiple_choice/physics.yaml' 2024-07-01T14:47:43,657 adding 'lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml' 2024-07-01T14:47:43,658 adding 'lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml' 2024-07-01T14:47:43,659 adding 'lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml' 2024-07-01T14:47:43,660 adding 'lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml' 2024-07-01T14:47:43,661 adding 'lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml' 2024-07-01T14:47:43,663 adding 'lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml' 2024-07-01T14:47:43,664 adding 'lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml' 2024-07-01T14:47:43,665 adding 'lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml' 2024-07-01T14:47:43,666 adding 'lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml' 2024-07-01T14:47:43,667 adding 'lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml' 2024-07-01T14:47:43,668 adding 'lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml' 2024-07-01T14:47:43,669 adding 'lm_eval/tasks/bigbench/multiple_choice/snarks.yaml' 2024-07-01T14:47:43,670 adding 'lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml' 2024-07-01T14:47:43,671 adding 'lm_eval/tasks/bigbench/multiple_choice/social_support.yaml' 2024-07-01T14:47:43,673 adding 'lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml' 2024-07-01T14:47:43,674 adding 'lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml' 2024-07-01T14:47:43,675 adding 'lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml' 2024-07-01T14:47:43,676 adding 'lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml' 2024-07-01T14:47:43,677 adding 'lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml' 2024-07-01T14:47:43,678 adding 'lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml' 2024-07-01T14:47:43,679 adding 'lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml' 2024-07-01T14:47:43,680 adding 'lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml' 2024-07-01T14:47:43,682 adding 'lm_eval/tasks/bigbench/multiple_choice/timedial.yaml' 2024-07-01T14:47:43,683 adding 'lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml' 2024-07-01T14:47:43,684 adding 'lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml' 2024-07-01T14:47:43,685 adding 'lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml' 2024-07-01T14:47:43,686 adding 'lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml' 2024-07-01T14:47:43,687 adding 'lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml' 2024-07-01T14:47:43,689 adding 'lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml' 2024-07-01T14:47:43,690 adding 'lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml' 2024-07-01T14:47:43,691 adding 'lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml' 2024-07-01T14:47:43,692 adding 'lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml' 2024-07-01T14:47:43,695 adding 'lm_eval/tasks/blimp/README.md' 2024-07-01T14:47:43,696 adding 'lm_eval/tasks/blimp/_template_yaml' 2024-07-01T14:47:43,697 adding 'lm_eval/tasks/blimp/adjunct_island.yaml' 2024-07-01T14:47:43,699 adding 'lm_eval/tasks/blimp/anaphor_gender_agreement.yaml' 2024-07-01T14:47:43,700 adding 'lm_eval/tasks/blimp/anaphor_number_agreement.yaml' 2024-07-01T14:47:43,701 adding 'lm_eval/tasks/blimp/animate_subject_passive.yaml' 2024-07-01T14:47:43,702 adding 'lm_eval/tasks/blimp/animate_subject_trans.yaml' 2024-07-01T14:47:43,703 adding 'lm_eval/tasks/blimp/causative.yaml' 2024-07-01T14:47:43,704 adding 'lm_eval/tasks/blimp/complex_NP_island.yaml' 2024-07-01T14:47:43,705 adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml' 2024-07-01T14:47:43,706 adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml' 2024-07-01T14:47:43,707 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml' 2024-07-01T14:47:43,708 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml' 2024-07-01T14:47:43,709 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml' 2024-07-01T14:47:43,710 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml' 2024-07-01T14:47:43,711 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml' 2024-07-01T14:47:43,712 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml' 2024-07-01T14:47:43,713 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml' 2024-07-01T14:47:43,714 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml' 2024-07-01T14:47:43,715 adding 'lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml' 2024-07-01T14:47:43,716 adding 'lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml' 2024-07-01T14:47:43,717 adding 'lm_eval/tasks/blimp/drop_argument.yaml' 2024-07-01T14:47:43,718 adding 'lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml' 2024-07-01T14:47:43,719 adding 'lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml' 2024-07-01T14:47:43,720 adding 'lm_eval/tasks/blimp/existential_there_object_raising.yaml' 2024-07-01T14:47:43,721 adding 'lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml' 2024-07-01T14:47:43,722 adding 'lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml' 2024-07-01T14:47:43,724 adding 'lm_eval/tasks/blimp/existential_there_subject_raising.yaml' 2024-07-01T14:47:43,725 adding 'lm_eval/tasks/blimp/expletive_it_object_raising.yaml' 2024-07-01T14:47:43,726 adding 'lm_eval/tasks/blimp/generate_configs.py' 2024-07-01T14:47:43,727 adding 'lm_eval/tasks/blimp/inchoative.yaml' 2024-07-01T14:47:43,728 adding 'lm_eval/tasks/blimp/intransitive.yaml' 2024-07-01T14:47:43,729 adding 'lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml' 2024-07-01T14:47:43,730 adding 'lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml' 2024-07-01T14:47:43,731 adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml' 2024-07-01T14:47:43,733 adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml' 2024-07-01T14:47:43,734 adding 'lm_eval/tasks/blimp/left_branch_island_echo_question.yaml' 2024-07-01T14:47:43,735 adding 'lm_eval/tasks/blimp/left_branch_island_simple_question.yaml' 2024-07-01T14:47:43,736 adding 'lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml' 2024-07-01T14:47:43,737 adding 'lm_eval/tasks/blimp/npi_present_1.yaml' 2024-07-01T14:47:43,738 adding 'lm_eval/tasks/blimp/npi_present_2.yaml' 2024-07-01T14:47:43,739 adding 'lm_eval/tasks/blimp/only_npi_licensor_present.yaml' 2024-07-01T14:47:43,740 adding 'lm_eval/tasks/blimp/only_npi_scope.yaml' 2024-07-01T14:47:43,741 adding 'lm_eval/tasks/blimp/passive_1.yaml' 2024-07-01T14:47:43,743 adding 'lm_eval/tasks/blimp/passive_2.yaml' 2024-07-01T14:47:43,744 adding 'lm_eval/tasks/blimp/principle_A_c_command.yaml' 2024-07-01T14:47:43,745 adding 'lm_eval/tasks/blimp/principle_A_case_1.yaml' 2024-07-01T14:47:43,746 adding 'lm_eval/tasks/blimp/principle_A_case_2.yaml' 2024-07-01T14:47:43,747 adding 'lm_eval/tasks/blimp/principle_A_domain_1.yaml' 2024-07-01T14:47:43,748 adding 'lm_eval/tasks/blimp/principle_A_domain_2.yaml' 2024-07-01T14:47:43,750 adding 'lm_eval/tasks/blimp/principle_A_domain_3.yaml' 2024-07-01T14:47:43,751 adding 'lm_eval/tasks/blimp/principle_A_reconstruction.yaml' 2024-07-01T14:47:43,752 adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml' 2024-07-01T14:47:43,753 adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml' 2024-07-01T14:47:43,754 adding 'lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml' 2024-07-01T14:47:43,755 adding 'lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml' 2024-07-01T14:47:43,756 adding 'lm_eval/tasks/blimp/sentential_subject_island.yaml' 2024-07-01T14:47:43,757 adding 'lm_eval/tasks/blimp/superlative_quantifiers_1.yaml' 2024-07-01T14:47:43,758 adding 'lm_eval/tasks/blimp/superlative_quantifiers_2.yaml' 2024-07-01T14:47:43,759 adding 'lm_eval/tasks/blimp/tough_vs_raising_1.yaml' 2024-07-01T14:47:43,760 adding 'lm_eval/tasks/blimp/tough_vs_raising_2.yaml' 2024-07-01T14:47:43,761 adding 'lm_eval/tasks/blimp/transitive.yaml' 2024-07-01T14:47:43,762 adding 'lm_eval/tasks/blimp/wh_island.yaml' 2024-07-01T14:47:43,763 adding 'lm_eval/tasks/blimp/wh_questions_object_gap.yaml' 2024-07-01T14:47:43,764 adding 'lm_eval/tasks/blimp/wh_questions_subject_gap.yaml' 2024-07-01T14:47:43,765 adding 'lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml' 2024-07-01T14:47:43,766 adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml' 2024-07-01T14:47:43,768 adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml' 2024-07-01T14:47:43,769 adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml' 2024-07-01T14:47:43,770 adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml' 2024-07-01T14:47:43,772 adding 'lm_eval/tasks/ceval/README.md' 2024-07-01T14:47:43,778 adding 'lm_eval/tasks/ceval/_default_ceval_yaml' 2024-07-01T14:47:43,779 adding 'lm_eval/tasks/ceval/_generate_configs.py' 2024-07-01T14:47:43,780 adding 'lm_eval/tasks/ceval/ceval-valid_accountant.yaml' 2024-07-01T14:47:43,782 adding 'lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml' 2024-07-01T14:47:43,783 adding 'lm_eval/tasks/ceval/ceval-valid_art_studies.yaml' 2024-07-01T14:47:43,784 adding 'lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml' 2024-07-01T14:47:43,785 adding 'lm_eval/tasks/ceval/ceval-valid_business_administration.yaml' 2024-07-01T14:47:43,786 adding 'lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml' 2024-07-01T14:47:43,787 adding 'lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml' 2024-07-01T14:47:43,788 adding 'lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml' 2024-07-01T14:47:43,790 adding 'lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml' 2024-07-01T14:47:43,791 adding 'lm_eval/tasks/ceval/ceval-valid_college_economics.yaml' 2024-07-01T14:47:43,792 adding 'lm_eval/tasks/ceval/ceval-valid_college_physics.yaml' 2024-07-01T14:47:43,793 adding 'lm_eval/tasks/ceval/ceval-valid_college_programming.yaml' 2024-07-01T14:47:43,794 adding 'lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml' 2024-07-01T14:47:43,795 adding 'lm_eval/tasks/ceval/ceval-valid_computer_network.yaml' 2024-07-01T14:47:43,797 adding 'lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml' 2024-07-01T14:47:43,798 adding 'lm_eval/tasks/ceval/ceval-valid_education_science.yaml' 2024-07-01T14:47:43,799 adding 'lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml' 2024-07-01T14:47:43,800 adding 'lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml' 2024-07-01T14:47:43,801 adding 'lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml' 2024-07-01T14:47:43,803 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml' 2024-07-01T14:47:43,804 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml' 2024-07-01T14:47:43,805 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml' 2024-07-01T14:47:43,806 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml' 2024-07-01T14:47:43,807 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml' 2024-07-01T14:47:43,808 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml' 2024-07-01T14:47:43,810 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml' 2024-07-01T14:47:43,810 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml' 2024-07-01T14:47:43,812 adding 'lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml' 2024-07-01T14:47:43,812 adding 'lm_eval/tasks/ceval/ceval-valid_law.yaml' 2024-07-01T14:47:43,814 adding 'lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml' 2024-07-01T14:47:43,815 adding 'lm_eval/tasks/ceval/ceval-valid_logic.yaml' 2024-07-01T14:47:43,816 adding 'lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml' 2024-07-01T14:47:43,817 adding 'lm_eval/tasks/ceval/ceval-valid_marxism.yaml' 2024-07-01T14:47:43,818 adding 'lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml' 2024-07-01T14:47:43,819 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml' 2024-07-01T14:47:43,820 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml' 2024-07-01T14:47:43,821 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml' 2024-07-01T14:47:43,822 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml' 2024-07-01T14:47:43,823 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml' 2024-07-01T14:47:43,824 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml' 2024-07-01T14:47:43,826 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml' 2024-07-01T14:47:43,827 adding 'lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml' 2024-07-01T14:47:43,828 adding 'lm_eval/tasks/ceval/ceval-valid_operating_system.yaml' 2024-07-01T14:47:43,829 adding 'lm_eval/tasks/ceval/ceval-valid_physician.yaml' 2024-07-01T14:47:43,830 adding 'lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml' 2024-07-01T14:47:43,831 adding 'lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml' 2024-07-01T14:47:43,832 adding 'lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml' 2024-07-01T14:47:43,833 adding 'lm_eval/tasks/ceval/ceval-valid_sports_science.yaml' 2024-07-01T14:47:43,834 adding 'lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml' 2024-07-01T14:47:43,835 adding 'lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml' 2024-07-01T14:47:43,837 adding 'lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml' 2024-07-01T14:47:43,838 adding 'lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml' 2024-07-01T14:47:43,841 adding 'lm_eval/tasks/cmmlu/README.md' 2024-07-01T14:47:43,842 adding 'lm_eval/tasks/cmmlu/_default_template_yaml' 2024-07-01T14:47:43,843 adding 'lm_eval/tasks/cmmlu/_generate_configs.py' 2024-07-01T14:47:43,845 adding 'lm_eval/tasks/cmmlu/cmmlu_default_agronomy.yaml' 2024-07-01T14:47:43,846 adding 'lm_eval/tasks/cmmlu/cmmlu_default_anatomy.yaml' 2024-07-01T14:47:43,847 adding 'lm_eval/tasks/cmmlu/cmmlu_default_ancient_chinese.yaml' 2024-07-01T14:47:43,848 adding 'lm_eval/tasks/cmmlu/cmmlu_default_arts.yaml' 2024-07-01T14:47:43,849 adding 'lm_eval/tasks/cmmlu/cmmlu_default_astronomy.yaml' 2024-07-01T14:47:43,851 adding 'lm_eval/tasks/cmmlu/cmmlu_default_business_ethics.yaml' 2024-07-01T14:47:43,852 adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_civil_service_exam.yaml' 2024-07-01T14:47:43,853 adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_driving_rule.yaml' 2024-07-01T14:47:43,855 adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_food_culture.yaml' 2024-07-01T14:47:43,856 adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_foreign_policy.yaml' 2024-07-01T14:47:43,857 adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_history.yaml' 2024-07-01T14:47:43,858 adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_literature.yaml' 2024-07-01T14:47:43,859 adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_teacher_qualification.yaml' 2024-07-01T14:47:43,861 adding 'lm_eval/tasks/cmmlu/cmmlu_default_clinical_knowledge.yaml' 2024-07-01T14:47:43,862 adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_actuarial_science.yaml' 2024-07-01T14:47:43,863 adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_education.yaml' 2024-07-01T14:47:43,864 adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_engineering_hydrology.yaml' 2024-07-01T14:47:43,865 adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_law.yaml' 2024-07-01T14:47:43,867 adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_mathematics.yaml' 2024-07-01T14:47:43,868 adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_medical_statistics.yaml' 2024-07-01T14:47:43,869 adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_medicine.yaml' 2024-07-01T14:47:43,870 adding 'lm_eval/tasks/cmmlu/cmmlu_default_computer_science.yaml' 2024-07-01T14:47:43,871 adding 'lm_eval/tasks/cmmlu/cmmlu_default_computer_security.yaml' 2024-07-01T14:47:43,872 adding 'lm_eval/tasks/cmmlu/cmmlu_default_conceptual_physics.yaml' 2024-07-01T14:47:43,873 adding 'lm_eval/tasks/cmmlu/cmmlu_default_construction_project_management.yaml' 2024-07-01T14:47:43,874 adding 'lm_eval/tasks/cmmlu/cmmlu_default_economics.yaml' 2024-07-01T14:47:43,875 adding 'lm_eval/tasks/cmmlu/cmmlu_default_education.yaml' 2024-07-01T14:47:43,876 adding 'lm_eval/tasks/cmmlu/cmmlu_default_electrical_engineering.yaml' 2024-07-01T14:47:43,877 adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_chinese.yaml' 2024-07-01T14:47:43,879 adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_commonsense.yaml' 2024-07-01T14:47:43,880 adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_information_and_technology.yaml' 2024-07-01T14:47:43,881 adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_mathematics.yaml' 2024-07-01T14:47:43,882 adding 'lm_eval/tasks/cmmlu/cmmlu_default_ethnology.yaml' 2024-07-01T14:47:43,883 adding 'lm_eval/tasks/cmmlu/cmmlu_default_food_science.yaml' 2024-07-01T14:47:43,884 adding 'lm_eval/tasks/cmmlu/cmmlu_default_genetics.yaml' 2024-07-01T14:47:43,885 adding 'lm_eval/tasks/cmmlu/cmmlu_default_global_facts.yaml' 2024-07-01T14:47:43,886 adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_biology.yaml' 2024-07-01T14:47:43,888 adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_chemistry.yaml' 2024-07-01T14:47:43,889 adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_geography.yaml' 2024-07-01T14:47:43,890 adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_mathematics.yaml' 2024-07-01T14:47:43,891 adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_physics.yaml' 2024-07-01T14:47:43,892 adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_politics.yaml' 2024-07-01T14:47:43,893 adding 'lm_eval/tasks/cmmlu/cmmlu_default_human_sexuality.yaml' 2024-07-01T14:47:43,895 adding 'lm_eval/tasks/cmmlu/cmmlu_default_international_law.yaml' 2024-07-01T14:47:43,896 adding 'lm_eval/tasks/cmmlu/cmmlu_default_journalism.yaml' 2024-07-01T14:47:43,897 adding 'lm_eval/tasks/cmmlu/cmmlu_default_jurisprudence.yaml' 2024-07-01T14:47:43,898 adding 'lm_eval/tasks/cmmlu/cmmlu_default_legal_and_moral_basis.yaml' 2024-07-01T14:47:43,899 adding 'lm_eval/tasks/cmmlu/cmmlu_default_logical.yaml' 2024-07-01T14:47:43,900 adding 'lm_eval/tasks/cmmlu/cmmlu_default_machine_learning.yaml' 2024-07-01T14:47:43,902 adding 'lm_eval/tasks/cmmlu/cmmlu_default_management.yaml' 2024-07-01T14:47:43,903 adding 'lm_eval/tasks/cmmlu/cmmlu_default_marketing.yaml' 2024-07-01T14:47:43,904 adding 'lm_eval/tasks/cmmlu/cmmlu_default_marxist_theory.yaml' 2024-07-01T14:47:43,905 adding 'lm_eval/tasks/cmmlu/cmmlu_default_modern_chinese.yaml' 2024-07-01T14:47:43,906 adding 'lm_eval/tasks/cmmlu/cmmlu_default_nutrition.yaml' 2024-07-01T14:47:43,908 adding 'lm_eval/tasks/cmmlu/cmmlu_default_philosophy.yaml' 2024-07-01T14:47:43,909 adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_accounting.yaml' 2024-07-01T14:47:43,910 adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_law.yaml' 2024-07-01T14:47:43,911 adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_medicine.yaml' 2024-07-01T14:47:43,912 adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml' 2024-07-01T14:47:43,914 adding 'lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml' 2024-07-01T14:47:43,915 adding 'lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml' 2024-07-01T14:47:43,916 adding 'lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml' 2024-07-01T14:47:43,917 adding 'lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml' 2024-07-01T14:47:43,918 adding 'lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml' 2024-07-01T14:47:43,919 adding 'lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml' 2024-07-01T14:47:43,920 adding 'lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml' 2024-07-01T14:47:43,921 adding 'lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml' 2024-07-01T14:47:43,923 adding 'lm_eval/tasks/code_x_glue/code-text/bleu.py' 2024-07-01T14:47:43,925 adding 'lm_eval/tasks/code_x_glue/code-text/go.yaml' 2024-07-01T14:47:43,926 adding 'lm_eval/tasks/code_x_glue/code-text/java.yaml' 2024-07-01T14:47:43,927 adding 'lm_eval/tasks/code_x_glue/code-text/javascript.yaml' 2024-07-01T14:47:43,928 adding 'lm_eval/tasks/code_x_glue/code-text/php.yaml' 2024-07-01T14:47:43,929 adding 'lm_eval/tasks/code_x_glue/code-text/python.yaml' 2024-07-01T14:47:43,930 adding 'lm_eval/tasks/code_x_glue/code-text/ruby.yaml' 2024-07-01T14:47:43,931 adding 'lm_eval/tasks/code_x_glue/code-text/utils.py' 2024-07-01T14:47:43,933 adding 'lm_eval/tasks/commonsense_qa/README.md' 2024-07-01T14:47:43,934 adding 'lm_eval/tasks/commonsense_qa/default.yaml' 2024-07-01T14:47:43,936 adding 'lm_eval/tasks/copal_id/README.md' 2024-07-01T14:47:43,937 adding 'lm_eval/tasks/copal_id/colloquial.yaml' 2024-07-01T14:47:43,938 adding 'lm_eval/tasks/copal_id/standard.yaml' 2024-07-01T14:47:43,939 adding 'lm_eval/tasks/copal_id/utils.py' 2024-07-01T14:47:43,941 adding 'lm_eval/tasks/coqa/README.md' 2024-07-01T14:47:43,942 adding 'lm_eval/tasks/coqa/default.yaml' 2024-07-01T14:47:43,943 adding 'lm_eval/tasks/coqa/utils.py' 2024-07-01T14:47:43,946 adding 'lm_eval/tasks/crows_pairs/README.md' 2024-07-01T14:47:43,947 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english.yaml' 2024-07-01T14:47:43,948 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml' 2024-07-01T14:47:43,949 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml' 2024-07-01T14:47:43,950 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml' 2024-07-01T14:47:43,952 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml' 2024-07-01T14:47:43,953 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml' 2024-07-01T14:47:43,954 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml' 2024-07-01T14:47:43,955 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml' 2024-07-01T14:47:43,956 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml' 2024-07-01T14:47:43,957 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml' 2024-07-01T14:47:43,958 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml' 2024-07-01T14:47:43,960 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french.yaml' 2024-07-01T14:47:43,961 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml' 2024-07-01T14:47:43,962 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml' 2024-07-01T14:47:43,963 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml' 2024-07-01T14:47:43,964 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml' 2024-07-01T14:47:43,965 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml' 2024-07-01T14:47:43,966 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml' 2024-07-01T14:47:43,967 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml' 2024-07-01T14:47:43,969 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml' 2024-07-01T14:47:43,970 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml' 2024-07-01T14:47:43,971 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml' 2024-07-01T14:47:43,972 adding 'lm_eval/tasks/crows_pairs/utils.py' 2024-07-01T14:47:43,974 adding 'lm_eval/tasks/csatqa/_default_csatqa_yaml' 2024-07-01T14:47:43,975 adding 'lm_eval/tasks/csatqa/_generate_configs.py' 2024-07-01T14:47:43,976 adding 'lm_eval/tasks/csatqa/csatqa_gr.yaml' 2024-07-01T14:47:43,977 adding 'lm_eval/tasks/csatqa/csatqa_li.yaml' 2024-07-01T14:47:43,978 adding 'lm_eval/tasks/csatqa/csatqa_rch.yaml' 2024-07-01T14:47:43,979 adding 'lm_eval/tasks/csatqa/csatqa_rcs.yaml' 2024-07-01T14:47:43,980 adding 'lm_eval/tasks/csatqa/csatqa_rcss.yaml' 2024-07-01T14:47:43,981 adding 'lm_eval/tasks/csatqa/csatqa_wr.yaml' 2024-07-01T14:47:43,982 adding 'lm_eval/tasks/csatqa/utils.py' 2024-07-01T14:47:43,984 adding 'lm_eval/tasks/drop/README.md' 2024-07-01T14:47:43,985 adding 'lm_eval/tasks/drop/default.yaml' 2024-07-01T14:47:43,986 adding 'lm_eval/tasks/drop/utils.py' 2024-07-01T14:47:43,988 adding 'lm_eval/tasks/eq_bench/README.md' 2024-07-01T14:47:43,989 adding 'lm_eval/tasks/eq_bench/default.yaml' 2024-07-01T14:47:43,991 adding 'lm_eval/tasks/eq_bench/utils.py' 2024-07-01T14:47:43,994 adding 'lm_eval/tasks/eus_exams/README.md' 2024-07-01T14:47:43,995 adding 'lm_eval/tasks/eus_exams/configs.py' 2024-07-01T14:47:43,996 adding 'lm_eval/tasks/eus_exams/eus_exams' 2024-07-01T14:47:43,997 adding 'lm_eval/tasks/eus_exams/eus_exams_es' 2024-07-01T14:47:43,998 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml' 2024-07-01T14:47:43,999 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml' 2024-07-01T14:47:44,000 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml' 2024-07-01T14:47:44,001 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml' 2024-07-01T14:47:44,002 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml' 2024-07-01T14:47:44,003 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml' 2024-07-01T14:47:44,005 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml' 2024-07-01T14:47:44,006 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml' 2024-07-01T14:47:44,007 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml' 2024-07-01T14:47:44,008 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml' 2024-07-01T14:47:44,009 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml' 2024-07-01T14:47:44,010 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml' 2024-07-01T14:47:44,011 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml' 2024-07-01T14:47:44,012 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml' 2024-07-01T14:47:44,013 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml' 2024-07-01T14:47:44,015 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml' 2024-07-01T14:47:44,016 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml' 2024-07-01T14:47:44,017 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml' 2024-07-01T14:47:44,018 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml' 2024-07-01T14:47:44,019 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml' 2024-07-01T14:47:44,020 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml' 2024-07-01T14:47:44,021 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml' 2024-07-01T14:47:44,022 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml' 2024-07-01T14:47:44,023 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml' 2024-07-01T14:47:44,025 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml' 2024-07-01T14:47:44,026 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml' 2024-07-01T14:47:44,027 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml' 2024-07-01T14:47:44,028 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml' 2024-07-01T14:47:44,029 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml' 2024-07-01T14:47:44,030 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml' 2024-07-01T14:47:44,031 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml' 2024-07-01T14:47:44,032 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml' 2024-07-01T14:47:44,033 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml' 2024-07-01T14:47:44,034 adding 'lm_eval/tasks/eus_exams/eus_exams_eu' 2024-07-01T14:47:44,035 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml' 2024-07-01T14:47:44,036 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml' 2024-07-01T14:47:44,037 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml' 2024-07-01T14:47:44,038 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml' 2024-07-01T14:47:44,039 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml' 2024-07-01T14:47:44,040 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml' 2024-07-01T14:47:44,041 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml' 2024-07-01T14:47:44,043 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml' 2024-07-01T14:47:44,044 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml' 2024-07-01T14:47:44,045 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml' 2024-07-01T14:47:44,046 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml' 2024-07-01T14:47:44,047 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml' 2024-07-01T14:47:44,048 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml' 2024-07-01T14:47:44,049 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml' 2024-07-01T14:47:44,050 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml' 2024-07-01T14:47:44,051 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml' 2024-07-01T14:47:44,052 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml' 2024-07-01T14:47:44,053 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml' 2024-07-01T14:47:44,054 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml' 2024-07-01T14:47:44,055 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml' 2024-07-01T14:47:44,056 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml' 2024-07-01T14:47:44,058 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml' 2024-07-01T14:47:44,059 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml' 2024-07-01T14:47:44,060 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml' 2024-07-01T14:47:44,061 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml' 2024-07-01T14:47:44,062 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml' 2024-07-01T14:47:44,063 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml' 2024-07-01T14:47:44,064 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml' 2024-07-01T14:47:44,065 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml' 2024-07-01T14:47:44,067 adding 'lm_eval/tasks/eus_exams/utils.py' 2024-07-01T14:47:44,068 adding 'lm_eval/tasks/eus_proficiency/README.md' 2024-07-01T14:47:44,070 adding 'lm_eval/tasks/eus_proficiency/eus_proficiency.yaml' 2024-07-01T14:47:44,071 adding 'lm_eval/tasks/eus_reading/README.md' 2024-07-01T14:47:44,073 adding 'lm_eval/tasks/eus_reading/eus_reading.yaml' 2024-07-01T14:47:44,074 adding 'lm_eval/tasks/eus_reading/utils.py' 2024-07-01T14:47:44,076 adding 'lm_eval/tasks/eus_trivia/README.md' 2024-07-01T14:47:44,077 adding 'lm_eval/tasks/eus_trivia/eus_trivia.yaml' 2024-07-01T14:47:44,078 adding 'lm_eval/tasks/eus_trivia/utils.py' 2024-07-01T14:47:44,080 adding 'lm_eval/tasks/fda/README.md' 2024-07-01T14:47:44,081 adding 'lm_eval/tasks/fda/fda.yaml' 2024-07-01T14:47:44,083 adding 'lm_eval/tasks/fda/task.py' 2024-07-01T14:47:44,085 adding 'lm_eval/tasks/fld/README.md' 2024-07-01T14:47:44,086 adding 'lm_eval/tasks/fld/fld_default.yaml' 2024-07-01T14:47:44,087 adding 'lm_eval/tasks/fld/fld_logical_formula_default.yaml' 2024-07-01T14:47:44,088 adding 'lm_eval/tasks/fld/fld_logical_formula_star.yaml' 2024-07-01T14:47:44,089 adding 'lm_eval/tasks/fld/fld_star.yaml' 2024-07-01T14:47:44,091 adding 'lm_eval/tasks/french_bench/README.md' 2024-07-01T14:47:44,092 adding 'lm_eval/tasks/french_bench/_default_template_yaml' 2024-07-01T14:47:44,093 adding 'lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml' 2024-07-01T14:47:44,094 adding 'lm_eval/tasks/french_bench/french_bench_boolqa.yaml' 2024-07-01T14:47:44,095 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2.yaml' 2024-07-01T14:47:44,096 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml' 2024-07-01T14:47:44,098 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml' 2024-07-01T14:47:44,099 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml' 2024-07-01T14:47:44,100 adding 'lm_eval/tasks/french_bench/french_bench_grammar.yaml' 2024-07-01T14:47:44,101 adding 'lm_eval/tasks/french_bench/french_bench_hellaswag.yaml' 2024-07-01T14:47:44,102 adding 'lm_eval/tasks/french_bench/french_bench_multifquad.yaml' 2024-07-01T14:47:44,104 adding 'lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml' 2024-07-01T14:47:44,105 adding 'lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml' 2024-07-01T14:47:44,106 adding 'lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml' 2024-07-01T14:47:44,107 adding 'lm_eval/tasks/french_bench/french_bench_reading_comp.yaml' 2024-07-01T14:47:44,108 adding 'lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml' 2024-07-01T14:47:44,109 adding 'lm_eval/tasks/french_bench/french_bench_trivia.yaml' 2024-07-01T14:47:44,110 adding 'lm_eval/tasks/french_bench/french_bench_vocab.yaml' 2024-07-01T14:47:44,111 adding 'lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml' 2024-07-01T14:47:44,113 adding 'lm_eval/tasks/french_bench/french_bench_xnli.yaml' 2024-07-01T14:47:44,114 adding 'lm_eval/tasks/french_bench/preprocess_wikitext.py' 2024-07-01T14:47:44,333 adding 'lm_eval/tasks/french_bench/utils.py' 2024-07-01T14:47:44,335 adding 'lm_eval/tasks/glianorex/README.md' 2024-07-01T14:47:44,336 adding 'lm_eval/tasks/glianorex/glianorex.yaml' 2024-07-01T14:47:44,338 adding 'lm_eval/tasks/glianorex/glianorex_en.yaml' 2024-07-01T14:47:44,339 adding 'lm_eval/tasks/glianorex/glianorex_fr.yaml' 2024-07-01T14:47:44,340 adding 'lm_eval/tasks/glianorex/preprocess_glianorex.py' 2024-07-01T14:47:44,342 adding 'lm_eval/tasks/glue/README.md' 2024-07-01T14:47:44,344 adding 'lm_eval/tasks/glue/cola/default.yaml' 2024-07-01T14:47:44,345 adding 'lm_eval/tasks/glue/mnli/default.yaml' 2024-07-01T14:47:44,346 adding 'lm_eval/tasks/glue/mnli/mismatch.yaml' 2024-07-01T14:47:44,347 adding 'lm_eval/tasks/glue/mnli/utils.py' 2024-07-01T14:47:44,349 adding 'lm_eval/tasks/glue/mrpc/default.yaml' 2024-07-01T14:47:44,351 adding 'lm_eval/tasks/glue/qnli/default.yaml' 2024-07-01T14:47:44,352 adding 'lm_eval/tasks/glue/qqp/default.yaml' 2024-07-01T14:47:44,354 adding 'lm_eval/tasks/glue/rte/default.yaml' 2024-07-01T14:47:44,356 adding 'lm_eval/tasks/glue/sst2/default.yaml' 2024-07-01T14:47:44,357 adding 'lm_eval/tasks/glue/wnli/default.yaml' 2024-07-01T14:47:44,359 adding 'lm_eval/tasks/gpqa/README.md' 2024-07-01T14:47:44,361 adding 'lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py' 2024-07-01T14:47:44,362 adding 'lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml' 2024-07-01T14:47:44,363 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml' 2024-07-01T14:47:44,365 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml' 2024-07-01T14:47:44,366 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml' 2024-07-01T14:47:44,367 adding 'lm_eval/tasks/gpqa/cot_n_shot/utils.py' 2024-07-01T14:47:44,369 adding 'lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py' 2024-07-01T14:47:44,370 adding 'lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml' 2024-07-01T14:47:44,371 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml' 2024-07-01T14:47:44,372 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml' 2024-07-01T14:47:44,373 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml' 2024-07-01T14:47:44,375 adding 'lm_eval/tasks/gpqa/cot_zeroshot/utils.py' 2024-07-01T14:47:44,376 adding 'lm_eval/tasks/gpqa/generative/_generate_configs.py' 2024-07-01T14:47:44,378 adding 'lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml' 2024-07-01T14:47:44,379 adding 'lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml' 2024-07-01T14:47:44,380 adding 'lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml' 2024-07-01T14:47:44,381 adding 'lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml' 2024-07-01T14:47:44,382 adding 'lm_eval/tasks/gpqa/generative/utils.py' 2024-07-01T14:47:44,384 adding 'lm_eval/tasks/gpqa/n_shot/_generate_configs.py' 2024-07-01T14:47:44,385 adding 'lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml' 2024-07-01T14:47:44,386 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml' 2024-07-01T14:47:44,387 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml' 2024-07-01T14:47:44,388 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml' 2024-07-01T14:47:44,389 adding 'lm_eval/tasks/gpqa/n_shot/utils.py' 2024-07-01T14:47:44,391 adding 'lm_eval/tasks/gpqa/zeroshot/_generate_configs.py' 2024-07-01T14:47:44,392 adding 'lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml' 2024-07-01T14:47:44,393 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml' 2024-07-01T14:47:44,394 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml' 2024-07-01T14:47:44,395 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml' 2024-07-01T14:47:44,396 adding 'lm_eval/tasks/gpqa/zeroshot/utils.py' 2024-07-01T14:47:44,398 adding 'lm_eval/tasks/gsm8k/README.md' 2024-07-01T14:47:44,399 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml' 2024-07-01T14:47:44,401 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml' 2024-07-01T14:47:44,402 adding 'lm_eval/tasks/gsm8k/gsm8k-cot.yaml' 2024-07-01T14:47:44,403 adding 'lm_eval/tasks/gsm8k/gsm8k.yaml' 2024-07-01T14:47:44,405 adding 'lm_eval/tasks/haerae/README.md' 2024-07-01T14:47:44,407 adding 'lm_eval/tasks/haerae/_default_haerae_yaml' 2024-07-01T14:47:44,408 adding 'lm_eval/tasks/haerae/haerae_gk.yaml' 2024-07-01T14:47:44,409 adding 'lm_eval/tasks/haerae/haerae_hi.yaml' 2024-07-01T14:47:44,410 adding 'lm_eval/tasks/haerae/haerae_lw.yaml' 2024-07-01T14:47:44,411 adding 'lm_eval/tasks/haerae/haerae_rw.yaml' 2024-07-01T14:47:44,412 adding 'lm_eval/tasks/haerae/haerae_sn.yaml' 2024-07-01T14:47:44,414 adding 'lm_eval/tasks/headqa/README.md' 2024-07-01T14:47:44,415 adding 'lm_eval/tasks/headqa/headqa_en.yaml' 2024-07-01T14:47:44,416 adding 'lm_eval/tasks/headqa/headqa_es.yaml' 2024-07-01T14:47:44,418 adding 'lm_eval/tasks/hellaswag/README.md' 2024-07-01T14:47:44,420 adding 'lm_eval/tasks/hellaswag/hellaswag.yaml' 2024-07-01T14:47:44,421 adding 'lm_eval/tasks/hellaswag/utils.py' 2024-07-01T14:47:44,423 adding 'lm_eval/tasks/hendrycks_ethics/README.md' 2024-07-01T14:47:44,424 adding 'lm_eval/tasks/hendrycks_ethics/commonsense.yaml' 2024-07-01T14:47:44,425 adding 'lm_eval/tasks/hendrycks_ethics/deontology.yaml' 2024-07-01T14:47:44,426 adding 'lm_eval/tasks/hendrycks_ethics/justice.yaml' 2024-07-01T14:47:44,427 adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml' 2024-07-01T14:47:44,429 adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml' 2024-07-01T14:47:44,430 adding 'lm_eval/tasks/hendrycks_ethics/utils.py' 2024-07-01T14:47:44,431 adding 'lm_eval/tasks/hendrycks_ethics/virtue.yaml' 2024-07-01T14:47:44,433 adding 'lm_eval/tasks/hendrycks_math/README.md' 2024-07-01T14:47:44,434 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math.yaml' 2024-07-01T14:47:44,435 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml' 2024-07-01T14:47:44,437 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml' 2024-07-01T14:47:44,438 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml' 2024-07-01T14:47:44,439 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml' 2024-07-01T14:47:44,440 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml' 2024-07-01T14:47:44,441 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml' 2024-07-01T14:47:44,442 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml' 2024-07-01T14:47:44,443 adding 'lm_eval/tasks/hendrycks_math/utils.py' 2024-07-01T14:47:44,445 adding 'lm_eval/tasks/ifeval/README.md' 2024-07-01T14:47:44,446 adding 'lm_eval/tasks/ifeval/ifeval.yaml' 2024-07-01T14:47:44,451 adding 'lm_eval/tasks/ifeval/instructions.py' 2024-07-01T14:47:44,453 adding 'lm_eval/tasks/ifeval/instructions_registry.py' 2024-07-01T14:47:44,457 adding 'lm_eval/tasks/ifeval/instructions_util.py' 2024-07-01T14:47:44,459 adding 'lm_eval/tasks/ifeval/utils.py' 2024-07-01T14:47:44,460 adding 'lm_eval/tasks/kmmlu/README.md' 2024-07-01T14:47:44,463 adding 'lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml' 2024-07-01T14:47:44,465 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml' 2024-07-01T14:47:44,467 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml' 2024-07-01T14:47:44,469 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml' 2024-07-01T14:47:44,470 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml' 2024-07-01T14:47:44,472 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml' 2024-07-01T14:47:44,474 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml' 2024-07-01T14:47:44,476 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml' 2024-07-01T14:47:44,478 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml' 2024-07-01T14:47:44,480 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml' 2024-07-01T14:47:44,482 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml' 2024-07-01T14:47:44,484 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml' 2024-07-01T14:47:44,486 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml' 2024-07-01T14:47:44,488 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml' 2024-07-01T14:47:44,489 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml' 2024-07-01T14:47:44,491 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml' 2024-07-01T14:47:44,493 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml' 2024-07-01T14:47:44,495 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml' 2024-07-01T14:47:44,497 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml' 2024-07-01T14:47:44,499 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml' 2024-07-01T14:47:44,500 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml' 2024-07-01T14:47:44,502 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml' 2024-07-01T14:47:44,504 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml' 2024-07-01T14:47:44,506 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml' 2024-07-01T14:47:44,508 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml' 2024-07-01T14:47:44,510 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml' 2024-07-01T14:47:44,512 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml' 2024-07-01T14:47:44,514 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml' 2024-07-01T14:47:44,516 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml' 2024-07-01T14:47:44,518 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml' 2024-07-01T14:47:44,520 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml' 2024-07-01T14:47:44,522 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml' 2024-07-01T14:47:44,524 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml' 2024-07-01T14:47:44,526 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml' 2024-07-01T14:47:44,527 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml' 2024-07-01T14:47:44,529 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml' 2024-07-01T14:47:44,531 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml' 2024-07-01T14:47:44,533 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml' 2024-07-01T14:47:44,535 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml' 2024-07-01T14:47:44,537 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml' 2024-07-01T14:47:44,539 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml' 2024-07-01T14:47:44,541 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml' 2024-07-01T14:47:44,543 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml' 2024-07-01T14:47:44,545 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml' 2024-07-01T14:47:44,547 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml' 2024-07-01T14:47:44,549 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml' 2024-07-01T14:47:44,551 adding 'lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml' 2024-07-01T14:47:44,553 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml' 2024-07-01T14:47:44,554 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml' 2024-07-01T14:47:44,555 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml' 2024-07-01T14:47:44,556 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml' 2024-07-01T14:47:44,557 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml' 2024-07-01T14:47:44,558 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml' 2024-07-01T14:47:44,559 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml' 2024-07-01T14:47:44,560 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml' 2024-07-01T14:47:44,561 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml' 2024-07-01T14:47:44,562 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml' 2024-07-01T14:47:44,563 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml' 2024-07-01T14:47:44,565 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml' 2024-07-01T14:47:44,566 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml' 2024-07-01T14:47:44,567 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml' 2024-07-01T14:47:44,568 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml' 2024-07-01T14:47:44,569 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml' 2024-07-01T14:47:44,570 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml' 2024-07-01T14:47:44,571 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml' 2024-07-01T14:47:44,572 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml' 2024-07-01T14:47:44,573 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml' 2024-07-01T14:47:44,575 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml' 2024-07-01T14:47:44,576 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml' 2024-07-01T14:47:44,577 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml' 2024-07-01T14:47:44,578 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml' 2024-07-01T14:47:44,579 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml' 2024-07-01T14:47:44,580 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml' 2024-07-01T14:47:44,581 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml' 2024-07-01T14:47:44,582 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml' 2024-07-01T14:47:44,583 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml' 2024-07-01T14:47:44,584 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml' 2024-07-01T14:47:44,585 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml' 2024-07-01T14:47:44,586 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml' 2024-07-01T14:47:44,587 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml' 2024-07-01T14:47:44,588 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml' 2024-07-01T14:47:44,589 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml' 2024-07-01T14:47:44,590 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml' 2024-07-01T14:47:44,591 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml' 2024-07-01T14:47:44,592 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml' 2024-07-01T14:47:44,593 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml' 2024-07-01T14:47:44,594 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml' 2024-07-01T14:47:44,595 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml' 2024-07-01T14:47:44,596 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml' 2024-07-01T14:47:44,597 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml' 2024-07-01T14:47:44,599 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml' 2024-07-01T14:47:44,600 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml' 2024-07-01T14:47:44,603 adding 'lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml' 2024-07-01T14:47:44,604 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml' 2024-07-01T14:47:44,605 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml' 2024-07-01T14:47:44,606 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml' 2024-07-01T14:47:44,607 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml' 2024-07-01T14:47:44,608 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml' 2024-07-01T14:47:44,610 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml' 2024-07-01T14:47:44,611 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml' 2024-07-01T14:47:44,612 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml' 2024-07-01T14:47:44,613 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml' 2024-07-01T14:47:44,614 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml' 2024-07-01T14:47:44,615 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml' 2024-07-01T14:47:44,617 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml' 2024-07-01T14:47:44,618 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml' 2024-07-01T14:47:44,619 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml' 2024-07-01T14:47:44,620 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml' 2024-07-01T14:47:44,621 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml' 2024-07-01T14:47:44,622 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml' 2024-07-01T14:47:44,623 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml' 2024-07-01T14:47:44,625 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml' 2024-07-01T14:47:44,626 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml' 2024-07-01T14:47:44,627 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml' 2024-07-01T14:47:44,628 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml' 2024-07-01T14:47:44,629 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml' 2024-07-01T14:47:44,630 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml' 2024-07-01T14:47:44,631 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml' 2024-07-01T14:47:44,632 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml' 2024-07-01T14:47:44,633 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml' 2024-07-01T14:47:44,634 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml' 2024-07-01T14:47:44,635 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml' 2024-07-01T14:47:44,637 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml' 2024-07-01T14:47:44,638 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml' 2024-07-01T14:47:44,639 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml' 2024-07-01T14:47:44,640 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml' 2024-07-01T14:47:44,641 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml' 2024-07-01T14:47:44,642 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml' 2024-07-01T14:47:44,643 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml' 2024-07-01T14:47:44,644 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml' 2024-07-01T14:47:44,646 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml' 2024-07-01T14:47:44,647 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml' 2024-07-01T14:47:44,648 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml' 2024-07-01T14:47:44,649 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml' 2024-07-01T14:47:44,650 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml' 2024-07-01T14:47:44,651 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml' 2024-07-01T14:47:44,652 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml' 2024-07-01T14:47:44,653 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml' 2024-07-01T14:47:44,656 adding 'lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml' 2024-07-01T14:47:44,657 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml' 2024-07-01T14:47:44,659 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml' 2024-07-01T14:47:44,660 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml' 2024-07-01T14:47:44,661 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml' 2024-07-01T14:47:44,662 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml' 2024-07-01T14:47:44,663 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml' 2024-07-01T14:47:44,664 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml' 2024-07-01T14:47:44,665 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml' 2024-07-01T14:47:44,667 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml' 2024-07-01T14:47:44,668 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml' 2024-07-01T14:47:44,669 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml' 2024-07-01T14:47:44,670 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml' 2024-07-01T14:47:44,671 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml' 2024-07-01T14:47:44,672 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml' 2024-07-01T14:47:44,673 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml' 2024-07-01T14:47:44,674 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml' 2024-07-01T14:47:44,675 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml' 2024-07-01T14:47:44,676 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml' 2024-07-01T14:47:44,677 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml' 2024-07-01T14:47:44,678 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml' 2024-07-01T14:47:44,679 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml' 2024-07-01T14:47:44,681 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml' 2024-07-01T14:47:44,682 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml' 2024-07-01T14:47:44,683 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml' 2024-07-01T14:47:44,684 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml' 2024-07-01T14:47:44,685 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml' 2024-07-01T14:47:44,686 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml' 2024-07-01T14:47:44,687 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml' 2024-07-01T14:47:44,688 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml' 2024-07-01T14:47:44,689 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml' 2024-07-01T14:47:44,690 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml' 2024-07-01T14:47:44,691 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml' 2024-07-01T14:47:44,692 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml' 2024-07-01T14:47:44,694 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml' 2024-07-01T14:47:44,695 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml' 2024-07-01T14:47:44,696 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml' 2024-07-01T14:47:44,697 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml' 2024-07-01T14:47:44,699 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml' 2024-07-01T14:47:44,700 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml' 2024-07-01T14:47:44,701 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml' 2024-07-01T14:47:44,702 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml' 2024-07-01T14:47:44,703 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml' 2024-07-01T14:47:44,704 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml' 2024-07-01T14:47:44,705 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml' 2024-07-01T14:47:44,707 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml' 2024-07-01T14:47:44,708 adding 'lm_eval/tasks/kobest/README.md' 2024-07-01T14:47:44,710 adding 'lm_eval/tasks/kobest/kobest_boolq.yaml' 2024-07-01T14:47:44,711 adding 'lm_eval/tasks/kobest/kobest_copa.yaml' 2024-07-01T14:47:44,712 adding 'lm_eval/tasks/kobest/kobest_hellaswag.yaml' 2024-07-01T14:47:44,713 adding 'lm_eval/tasks/kobest/kobest_sentineg.yaml' 2024-07-01T14:47:44,715 adding 'lm_eval/tasks/kobest/kobest_wic.yaml' 2024-07-01T14:47:44,716 adding 'lm_eval/tasks/kobest/utils.py' 2024-07-01T14:47:44,718 adding 'lm_eval/tasks/kormedmcqa/README.md' 2024-07-01T14:47:44,719 adding 'lm_eval/tasks/kormedmcqa/kormedmcqa_doctor.yaml' 2024-07-01T14:47:44,720 adding 'lm_eval/tasks/kormedmcqa/kormedmcqa_nurse.yaml' 2024-07-01T14:47:44,722 adding 'lm_eval/tasks/kormedmcqa/kormedmcqa_pharm.yaml' 2024-07-01T14:47:44,723 adding 'lm_eval/tasks/lambada/README.md' 2024-07-01T14:47:44,725 adding 'lm_eval/tasks/lambada/lambada_openai.yaml' 2024-07-01T14:47:44,726 adding 'lm_eval/tasks/lambada/lambada_standard.yaml' 2024-07-01T14:47:44,727 adding 'lm_eval/tasks/lambada_cloze/README.md' 2024-07-01T14:47:44,728 adding 'lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml' 2024-07-01T14:47:44,730 adding 'lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml' 2024-07-01T14:47:44,731 adding 'lm_eval/tasks/lambada_multilingual/README.md' 2024-07-01T14:47:44,732 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml' 2024-07-01T14:47:44,734 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml' 2024-07-01T14:47:44,735 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml' 2024-07-01T14:47:44,736 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml' 2024-07-01T14:47:44,737 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml' 2024-07-01T14:47:44,739 adding 'lm_eval/tasks/lambada_multilingual_stablelm/README.md' 2024-07-01T14:47:44,740 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml' 2024-07-01T14:47:44,741 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml' 2024-07-01T14:47:44,742 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml' 2024-07-01T14:47:44,743 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml' 2024-07-01T14:47:44,744 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml' 2024-07-01T14:47:44,745 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml' 2024-07-01T14:47:44,746 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml' 2024-07-01T14:47:44,748 adding 'lm_eval/tasks/logiqa/README.md' 2024-07-01T14:47:44,749 adding 'lm_eval/tasks/logiqa/logiqa.yaml' 2024-07-01T14:47:44,751 adding 'lm_eval/tasks/logiqa/utils_logiqa.py' 2024-07-01T14:47:44,753 adding 'lm_eval/tasks/logiqa2/README.md' 2024-07-01T14:47:44,754 adding 'lm_eval/tasks/logiqa2/logieval.yaml' 2024-07-01T14:47:44,755 adding 'lm_eval/tasks/logiqa2/logiqa2.yaml' 2024-07-01T14:47:44,756 adding 'lm_eval/tasks/logiqa2/utils_logiqa2.py' 2024-07-01T14:47:44,758 adding 'lm_eval/tasks/mathqa/README.md' 2024-07-01T14:47:44,759 adding 'lm_eval/tasks/mathqa/mathqa.yaml' 2024-07-01T14:47:44,760 adding 'lm_eval/tasks/mathqa/utils.py' 2024-07-01T14:47:44,762 adding 'lm_eval/tasks/mc_taco/README.md' 2024-07-01T14:47:44,763 adding 'lm_eval/tasks/mc_taco/default.yaml' 2024-07-01T14:47:44,765 adding 'lm_eval/tasks/medmcqa/medmcqa.yaml' 2024-07-01T14:47:44,766 adding 'lm_eval/tasks/medmcqa/utils_medmcqa.py' 2024-07-01T14:47:44,768 adding 'lm_eval/tasks/medqa/medqa.yaml' 2024-07-01T14:47:44,769 adding 'lm_eval/tasks/medqa/preprocess_medqa.py' 2024-07-01T14:47:44,771 adding 'lm_eval/tasks/mgsm/README.md' 2024-07-01T14:47:44,772 adding 'lm_eval/tasks/mgsm/gen_yaml.sh' 2024-07-01T14:47:44,774 adding 'lm_eval/tasks/mgsm/utils.py' 2024-07-01T14:47:44,776 adding 'lm_eval/tasks/mgsm/direct/direct_yaml' 2024-07-01T14:47:44,777 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml' 2024-07-01T14:47:44,778 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml' 2024-07-01T14:47:44,779 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml' 2024-07-01T14:47:44,781 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml' 2024-07-01T14:47:44,782 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml' 2024-07-01T14:47:44,783 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml' 2024-07-01T14:47:44,784 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml' 2024-07-01T14:47:44,785 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml' 2024-07-01T14:47:44,787 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml' 2024-07-01T14:47:44,788 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml' 2024-07-01T14:47:44,789 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml' 2024-07-01T14:47:44,791 adding 'lm_eval/tasks/mgsm/en_cot/cot_yaml' 2024-07-01T14:47:44,792 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml' 2024-07-01T14:47:44,793 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml' 2024-07-01T14:47:44,794 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml' 2024-07-01T14:47:44,795 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml' 2024-07-01T14:47:44,796 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml' 2024-07-01T14:47:44,798 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml' 2024-07-01T14:47:44,799 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml' 2024-07-01T14:47:44,800 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml' 2024-07-01T14:47:44,801 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml' 2024-07-01T14:47:44,802 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml' 2024-07-01T14:47:44,803 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml' 2024-07-01T14:47:44,805 adding 'lm_eval/tasks/mgsm/native_cot/cot_yaml' 2024-07-01T14:47:44,806 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml' 2024-07-01T14:47:44,807 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml' 2024-07-01T14:47:44,808 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml' 2024-07-01T14:47:44,809 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml' 2024-07-01T14:47:44,811 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml' 2024-07-01T14:47:44,812 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml' 2024-07-01T14:47:44,813 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml' 2024-07-01T14:47:44,814 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml' 2024-07-01T14:47:44,815 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml' 2024-07-01T14:47:44,816 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml' 2024-07-01T14:47:44,817 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml' 2024-07-01T14:47:44,819 adding 'lm_eval/tasks/minerva_math/README.md' 2024-07-01T14:47:44,821 adding 'lm_eval/tasks/minerva_math/minerva_math_algebra.yaml' 2024-07-01T14:47:44,822 adding 'lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml' 2024-07-01T14:47:44,823 adding 'lm_eval/tasks/minerva_math/minerva_math_geometry.yaml' 2024-07-01T14:47:44,824 adding 'lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml' 2024-07-01T14:47:44,825 adding 'lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml' 2024-07-01T14:47:44,826 adding 'lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml' 2024-07-01T14:47:44,827 adding 'lm_eval/tasks/minerva_math/minerva_math_precalc.yaml' 2024-07-01T14:47:44,829 adding 'lm_eval/tasks/minerva_math/utils.py' 2024-07-01T14:47:44,831 adding 'lm_eval/tasks/mmlu/_generate_configs.py' 2024-07-01T14:47:44,834 adding 'lm_eval/tasks/mmlu/continuation/_continuation_template_yaml' 2024-07-01T14:47:44,836 adding 'lm_eval/tasks/mmlu/continuation/_mmlu.yaml' 2024-07-01T14:47:44,838 adding 'lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml' 2024-07-01T14:47:44,839 adding 'lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml' 2024-07-01T14:47:44,841 adding 'lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml' 2024-07-01T14:47:44,843 adding 'lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml' 2024-07-01T14:47:44,844 adding 'lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml' 2024-07-01T14:47:44,845 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml' 2024-07-01T14:47:44,846 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml' 2024-07-01T14:47:44,848 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml' 2024-07-01T14:47:44,849 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml' 2024-07-01T14:47:44,850 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml' 2024-07-01T14:47:44,851 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml' 2024-07-01T14:47:44,852 adding 'lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml' 2024-07-01T14:47:44,854 adding 'lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml' 2024-07-01T14:47:44,855 adding 'lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml' 2024-07-01T14:47:44,856 adding 'lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml' 2024-07-01T14:47:44,857 adding 'lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml' 2024-07-01T14:47:44,858 adding 'lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml' 2024-07-01T14:47:44,859 adding 'lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml' 2024-07-01T14:47:44,860 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml' 2024-07-01T14:47:44,861 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml' 2024-07-01T14:47:44,862 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml' 2024-07-01T14:47:44,863 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml' 2024-07-01T14:47:44,864 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml' 2024-07-01T14:47:44,866 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml' 2024-07-01T14:47:44,867 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml' 2024-07-01T14:47:44,868 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml' 2024-07-01T14:47:44,869 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml' 2024-07-01T14:47:44,870 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml' 2024-07-01T14:47:44,871 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml' 2024-07-01T14:47:44,873 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml' 2024-07-01T14:47:44,874 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml' 2024-07-01T14:47:44,875 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml' 2024-07-01T14:47:44,876 adding 'lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml' 2024-07-01T14:47:44,877 adding 'lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml' 2024-07-01T14:47:44,878 adding 'lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml' 2024-07-01T14:47:44,879 adding 'lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml' 2024-07-01T14:47:44,881 adding 'lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml' 2024-07-01T14:47:44,882 adding 'lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml' 2024-07-01T14:47:44,883 adding 'lm_eval/tasks/mmlu/continuation/mmlu_management.yaml' 2024-07-01T14:47:44,884 adding 'lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml' 2024-07-01T14:47:44,886 adding 'lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml' 2024-07-01T14:47:44,887 adding 'lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml' 2024-07-01T14:47:44,888 adding 'lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml' 2024-07-01T14:47:44,889 adding 'lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml' 2024-07-01T14:47:44,890 adding 'lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml' 2024-07-01T14:47:44,891 adding 'lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml' 2024-07-01T14:47:44,892 adding 'lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml' 2024-07-01T14:47:44,894 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml' 2024-07-01T14:47:44,895 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml' 2024-07-01T14:47:44,896 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml' 2024-07-01T14:47:44,897 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml' 2024-07-01T14:47:44,898 adding 'lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml' 2024-07-01T14:47:44,899 adding 'lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml' 2024-07-01T14:47:44,901 adding 'lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml' 2024-07-01T14:47:44,902 adding 'lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml' 2024-07-01T14:47:44,903 adding 'lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml' 2024-07-01T14:47:44,904 adding 'lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml' 2024-07-01T14:47:44,906 adding 'lm_eval/tasks/mmlu/default/_default_template_yaml' 2024-07-01T14:47:44,907 adding 'lm_eval/tasks/mmlu/default/_mmlu.yaml' 2024-07-01T14:47:44,908 adding 'lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml' 2024-07-01T14:47:44,910 adding 'lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml' 2024-07-01T14:47:44,911 adding 'lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml' 2024-07-01T14:47:44,912 adding 'lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml' 2024-07-01T14:47:44,913 adding 'lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml' 2024-07-01T14:47:44,914 adding 'lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml' 2024-07-01T14:47:44,915 adding 'lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml' 2024-07-01T14:47:44,916 adding 'lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml' 2024-07-01T14:47:44,917 adding 'lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml' 2024-07-01T14:47:44,918 adding 'lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml' 2024-07-01T14:47:44,919 adding 'lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml' 2024-07-01T14:47:44,920 adding 'lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml' 2024-07-01T14:47:44,921 adding 'lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml' 2024-07-01T14:47:44,922 adding 'lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml' 2024-07-01T14:47:44,923 adding 'lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml' 2024-07-01T14:47:44,924 adding 'lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml' 2024-07-01T14:47:44,925 adding 'lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml' 2024-07-01T14:47:44,927 adding 'lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml' 2024-07-01T14:47:44,928 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml' 2024-07-01T14:47:44,929 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml' 2024-07-01T14:47:44,930 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml' 2024-07-01T14:47:44,931 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml' 2024-07-01T14:47:44,932 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml' 2024-07-01T14:47:44,934 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml' 2024-07-01T14:47:44,935 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml' 2024-07-01T14:47:44,936 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml' 2024-07-01T14:47:44,937 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml' 2024-07-01T14:47:44,939 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml' 2024-07-01T14:47:44,940 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml' 2024-07-01T14:47:44,941 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml' 2024-07-01T14:47:44,942 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml' 2024-07-01T14:47:44,943 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml' 2024-07-01T14:47:44,945 adding 'lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml' 2024-07-01T14:47:44,946 adding 'lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml' 2024-07-01T14:47:44,947 adding 'lm_eval/tasks/mmlu/default/mmlu_international_law.yaml' 2024-07-01T14:47:44,948 adding 'lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml' 2024-07-01T14:47:44,949 adding 'lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml' 2024-07-01T14:47:44,951 adding 'lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml' 2024-07-01T14:47:44,952 adding 'lm_eval/tasks/mmlu/default/mmlu_management.yaml' 2024-07-01T14:47:44,953 adding 'lm_eval/tasks/mmlu/default/mmlu_marketing.yaml' 2024-07-01T14:47:44,954 adding 'lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml' 2024-07-01T14:47:44,955 adding 'lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml' 2024-07-01T14:47:44,956 adding 'lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml' 2024-07-01T14:47:44,957 adding 'lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml' 2024-07-01T14:47:44,958 adding 'lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml' 2024-07-01T14:47:44,960 adding 'lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml' 2024-07-01T14:47:44,961 adding 'lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml' 2024-07-01T14:47:44,962 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml' 2024-07-01T14:47:44,964 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml' 2024-07-01T14:47:44,965 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml' 2024-07-01T14:47:44,966 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml' 2024-07-01T14:47:44,968 adding 'lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml' 2024-07-01T14:47:44,969 adding 'lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml' 2024-07-01T14:47:44,970 adding 'lm_eval/tasks/mmlu/default/mmlu_sociology.yaml' 2024-07-01T14:47:44,971 adding 'lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml' 2024-07-01T14:47:44,973 adding 'lm_eval/tasks/mmlu/default/mmlu_virology.yaml' 2024-07-01T14:47:44,974 adding 'lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml' 2024-07-01T14:47:45,014 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json' 2024-07-01T14:47:45,017 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml' 2024-07-01T14:47:45,019 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml' 2024-07-01T14:47:45,020 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml' 2024-07-01T14:47:45,022 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml' 2024-07-01T14:47:45,024 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml' 2024-07-01T14:47:45,026 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml' 2024-07-01T14:47:45,027 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml' 2024-07-01T14:47:45,029 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml' 2024-07-01T14:47:45,031 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml' 2024-07-01T14:47:45,033 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml' 2024-07-01T14:47:45,035 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml' 2024-07-01T14:47:45,036 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml' 2024-07-01T14:47:45,038 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml' 2024-07-01T14:47:45,039 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml' 2024-07-01T14:47:45,041 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml' 2024-07-01T14:47:45,043 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml' 2024-07-01T14:47:45,045 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml' 2024-07-01T14:47:45,046 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml' 2024-07-01T14:47:45,048 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml' 2024-07-01T14:47:45,050 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml' 2024-07-01T14:47:45,051 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml' 2024-07-01T14:47:45,053 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml' 2024-07-01T14:47:45,055 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml' 2024-07-01T14:47:45,058 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml' 2024-07-01T14:47:45,060 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml' 2024-07-01T14:47:45,061 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml' 2024-07-01T14:47:45,063 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml' 2024-07-01T14:47:45,065 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml' 2024-07-01T14:47:45,066 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml' 2024-07-01T14:47:45,068 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml' 2024-07-01T14:47:45,070 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml' 2024-07-01T14:47:45,072 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml' 2024-07-01T14:47:45,074 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml' 2024-07-01T14:47:45,076 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml' 2024-07-01T14:47:45,078 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml' 2024-07-01T14:47:45,079 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml' 2024-07-01T14:47:45,081 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml' 2024-07-01T14:47:45,083 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml' 2024-07-01T14:47:45,085 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml' 2024-07-01T14:47:45,086 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml' 2024-07-01T14:47:45,088 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml' 2024-07-01T14:47:45,090 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml' 2024-07-01T14:47:45,091 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml' 2024-07-01T14:47:45,093 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml' 2024-07-01T14:47:45,095 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml' 2024-07-01T14:47:45,097 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml' 2024-07-01T14:47:45,098 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml' 2024-07-01T14:47:45,100 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml' 2024-07-01T14:47:45,102 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml' 2024-07-01T14:47:45,104 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml' 2024-07-01T14:47:45,106 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml' 2024-07-01T14:47:45,107 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml' 2024-07-01T14:47:45,109 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml' 2024-07-01T14:47:45,110 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml' 2024-07-01T14:47:45,112 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml' 2024-07-01T14:47:45,113 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml' 2024-07-01T14:47:45,114 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml' 2024-07-01T14:47:45,116 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml' 2024-07-01T14:47:45,117 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml' 2024-07-01T14:47:45,120 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml' 2024-07-01T14:47:45,121 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml' 2024-07-01T14:47:45,122 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml' 2024-07-01T14:47:45,123 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml' 2024-07-01T14:47:45,124 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml' 2024-07-01T14:47:45,125 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml' 2024-07-01T14:47:45,127 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml' 2024-07-01T14:47:45,128 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml' 2024-07-01T14:47:45,129 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml' 2024-07-01T14:47:45,130 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml' 2024-07-01T14:47:45,131 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml' 2024-07-01T14:47:45,132 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml' 2024-07-01T14:47:45,133 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml' 2024-07-01T14:47:45,134 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml' 2024-07-01T14:47:45,135 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml' 2024-07-01T14:47:45,136 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml' 2024-07-01T14:47:45,137 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml' 2024-07-01T14:47:45,138 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml' 2024-07-01T14:47:45,139 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml' 2024-07-01T14:47:45,140 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml' 2024-07-01T14:47:45,141 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml' 2024-07-01T14:47:45,143 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml' 2024-07-01T14:47:45,144 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml' 2024-07-01T14:47:45,145 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml' 2024-07-01T14:47:45,146 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml' 2024-07-01T14:47:45,147 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml' 2024-07-01T14:47:45,148 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml' 2024-07-01T14:47:45,149 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml' 2024-07-01T14:47:45,150 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml' 2024-07-01T14:47:45,151 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml' 2024-07-01T14:47:45,152 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml' 2024-07-01T14:47:45,154 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml' 2024-07-01T14:47:45,155 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml' 2024-07-01T14:47:45,156 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml' 2024-07-01T14:47:45,157 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml' 2024-07-01T14:47:45,158 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml' 2024-07-01T14:47:45,159 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml' 2024-07-01T14:47:45,161 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml' 2024-07-01T14:47:45,162 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml' 2024-07-01T14:47:45,163 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml' 2024-07-01T14:47:45,164 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml' 2024-07-01T14:47:45,165 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml' 2024-07-01T14:47:45,166 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml' 2024-07-01T14:47:45,167 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml' 2024-07-01T14:47:45,169 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml' 2024-07-01T14:47:45,170 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml' 2024-07-01T14:47:45,171 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml' 2024-07-01T14:47:45,172 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml' 2024-07-01T14:47:45,173 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml' 2024-07-01T14:47:45,175 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml' 2024-07-01T14:47:45,176 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml' 2024-07-01T14:47:45,177 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml' 2024-07-01T14:47:45,178 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml' 2024-07-01T14:47:45,179 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml' 2024-07-01T14:47:45,181 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml' 2024-07-01T14:47:45,182 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml' 2024-07-01T14:47:45,183 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml' 2024-07-01T14:47:45,184 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml' 2024-07-01T14:47:45,185 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml' 2024-07-01T14:47:45,186 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py' 2024-07-01T14:47:45,189 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml' 2024-07-01T14:47:45,191 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml' 2024-07-01T14:47:45,192 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml' 2024-07-01T14:47:45,193 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml' 2024-07-01T14:47:45,194 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml' 2024-07-01T14:47:45,195 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml' 2024-07-01T14:47:45,196 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml' 2024-07-01T14:47:45,197 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml' 2024-07-01T14:47:45,198 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml' 2024-07-01T14:47:45,199 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml' 2024-07-01T14:47:45,201 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml' 2024-07-01T14:47:45,202 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml' 2024-07-01T14:47:45,203 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml' 2024-07-01T14:47:45,204 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml' 2024-07-01T14:47:45,205 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml' 2024-07-01T14:47:45,206 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml' 2024-07-01T14:47:45,208 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml' 2024-07-01T14:47:45,209 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml' 2024-07-01T14:47:45,210 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml' 2024-07-01T14:47:45,211 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml' 2024-07-01T14:47:45,212 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml' 2024-07-01T14:47:45,214 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml' 2024-07-01T14:47:45,215 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml' 2024-07-01T14:47:45,216 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml' 2024-07-01T14:47:45,217 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml' 2024-07-01T14:47:45,218 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml' 2024-07-01T14:47:45,219 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml' 2024-07-01T14:47:45,220 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml' 2024-07-01T14:47:45,222 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml' 2024-07-01T14:47:45,223 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml' 2024-07-01T14:47:45,224 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml' 2024-07-01T14:47:45,225 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml' 2024-07-01T14:47:45,226 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml' 2024-07-01T14:47:45,227 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml' 2024-07-01T14:47:45,228 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml' 2024-07-01T14:47:45,230 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml' 2024-07-01T14:47:45,231 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml' 2024-07-01T14:47:45,232 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml' 2024-07-01T14:47:45,233 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml' 2024-07-01T14:47:45,234 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml' 2024-07-01T14:47:45,235 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml' 2024-07-01T14:47:45,236 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml' 2024-07-01T14:47:45,237 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml' 2024-07-01T14:47:45,239 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml' 2024-07-01T14:47:45,240 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml' 2024-07-01T14:47:45,241 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml' 2024-07-01T14:47:45,242 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml' 2024-07-01T14:47:45,243 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml' 2024-07-01T14:47:45,244 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml' 2024-07-01T14:47:45,245 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml' 2024-07-01T14:47:45,247 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml' 2024-07-01T14:47:45,248 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml' 2024-07-01T14:47:45,249 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml' 2024-07-01T14:47:45,250 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml' 2024-07-01T14:47:45,251 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml' 2024-07-01T14:47:45,252 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml' 2024-07-01T14:47:45,254 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml' 2024-07-01T14:47:45,255 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml' 2024-07-01T14:47:45,256 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml' 2024-07-01T14:47:45,257 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py' 2024-07-01T14:47:45,260 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml' 2024-07-01T14:47:45,261 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml' 2024-07-01T14:47:45,262 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml' 2024-07-01T14:47:45,263 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml' 2024-07-01T14:47:45,264 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml' 2024-07-01T14:47:45,266 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml' 2024-07-01T14:47:45,267 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml' 2024-07-01T14:47:45,268 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml' 2024-07-01T14:47:45,269 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml' 2024-07-01T14:47:45,270 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml' 2024-07-01T14:47:45,271 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml' 2024-07-01T14:47:45,272 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml' 2024-07-01T14:47:45,273 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml' 2024-07-01T14:47:45,274 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml' 2024-07-01T14:47:45,276 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml' 2024-07-01T14:47:45,277 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml' 2024-07-01T14:47:45,278 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml' 2024-07-01T14:47:45,279 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml' 2024-07-01T14:47:45,280 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml' 2024-07-01T14:47:45,281 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml' 2024-07-01T14:47:45,282 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml' 2024-07-01T14:47:45,283 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml' 2024-07-01T14:47:45,285 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml' 2024-07-01T14:47:45,286 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml' 2024-07-01T14:47:45,287 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml' 2024-07-01T14:47:45,288 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml' 2024-07-01T14:47:45,289 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml' 2024-07-01T14:47:45,290 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml' 2024-07-01T14:47:45,291 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml' 2024-07-01T14:47:45,293 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml' 2024-07-01T14:47:45,294 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml' 2024-07-01T14:47:45,295 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml' 2024-07-01T14:47:45,296 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml' 2024-07-01T14:47:45,297 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml' 2024-07-01T14:47:45,298 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml' 2024-07-01T14:47:45,299 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml' 2024-07-01T14:47:45,301 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml' 2024-07-01T14:47:45,302 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml' 2024-07-01T14:47:45,303 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml' 2024-07-01T14:47:45,304 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml' 2024-07-01T14:47:45,305 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml' 2024-07-01T14:47:45,306 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml' 2024-07-01T14:47:45,308 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml' 2024-07-01T14:47:45,309 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml' 2024-07-01T14:47:45,310 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml' 2024-07-01T14:47:45,311 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml' 2024-07-01T14:47:45,313 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml' 2024-07-01T14:47:45,314 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml' 2024-07-01T14:47:45,315 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml' 2024-07-01T14:47:45,316 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml' 2024-07-01T14:47:45,317 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml' 2024-07-01T14:47:45,318 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml' 2024-07-01T14:47:45,319 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml' 2024-07-01T14:47:45,320 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml' 2024-07-01T14:47:45,321 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml' 2024-07-01T14:47:45,322 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml' 2024-07-01T14:47:45,323 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml' 2024-07-01T14:47:45,324 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml' 2024-07-01T14:47:45,325 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml' 2024-07-01T14:47:45,328 adding 'lm_eval/tasks/mmlu/generative/_default_template_yaml' 2024-07-01T14:47:45,329 adding 'lm_eval/tasks/mmlu/generative/_mmlu.yaml' 2024-07-01T14:47:45,330 adding 'lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml' 2024-07-01T14:47:45,331 adding 'lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml' 2024-07-01T14:47:45,332 adding 'lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml' 2024-07-01T14:47:45,333 adding 'lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml' 2024-07-01T14:47:45,334 adding 'lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml' 2024-07-01T14:47:45,336 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml' 2024-07-01T14:47:45,337 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml' 2024-07-01T14:47:45,338 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml' 2024-07-01T14:47:45,339 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml' 2024-07-01T14:47:45,340 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml' 2024-07-01T14:47:45,341 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml' 2024-07-01T14:47:45,343 adding 'lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml' 2024-07-01T14:47:45,344 adding 'lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml' 2024-07-01T14:47:45,345 adding 'lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml' 2024-07-01T14:47:45,346 adding 'lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml' 2024-07-01T14:47:45,347 adding 'lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml' 2024-07-01T14:47:45,348 adding 'lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml' 2024-07-01T14:47:45,350 adding 'lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml' 2024-07-01T14:47:45,351 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml' 2024-07-01T14:47:45,352 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml' 2024-07-01T14:47:45,353 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml' 2024-07-01T14:47:45,354 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml' 2024-07-01T14:47:45,356 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml' 2024-07-01T14:47:45,357 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml' 2024-07-01T14:47:45,358 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml' 2024-07-01T14:47:45,359 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml' 2024-07-01T14:47:45,360 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml' 2024-07-01T14:47:45,362 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml' 2024-07-01T14:47:45,363 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml' 2024-07-01T14:47:45,364 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml' 2024-07-01T14:47:45,365 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml' 2024-07-01T14:47:45,366 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml' 2024-07-01T14:47:45,367 adding 'lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml' 2024-07-01T14:47:45,368 adding 'lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml' 2024-07-01T14:47:45,369 adding 'lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml' 2024-07-01T14:47:45,370 adding 'lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml' 2024-07-01T14:47:45,371 adding 'lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml' 2024-07-01T14:47:45,373 adding 'lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml' 2024-07-01T14:47:45,374 adding 'lm_eval/tasks/mmlu/generative/mmlu_management.yaml' 2024-07-01T14:47:45,375 adding 'lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml' 2024-07-01T14:47:45,376 adding 'lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml' 2024-07-01T14:47:45,377 adding 'lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml' 2024-07-01T14:47:45,379 adding 'lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml' 2024-07-01T14:47:45,380 adding 'lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml' 2024-07-01T14:47:45,381 adding 'lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml' 2024-07-01T14:47:45,382 adding 'lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml' 2024-07-01T14:47:45,383 adding 'lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml' 2024-07-01T14:47:45,384 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml' 2024-07-01T14:47:45,385 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml' 2024-07-01T14:47:45,387 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml' 2024-07-01T14:47:45,388 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml' 2024-07-01T14:47:45,389 adding 'lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml' 2024-07-01T14:47:45,390 adding 'lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml' 2024-07-01T14:47:45,391 adding 'lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml' 2024-07-01T14:47:45,393 adding 'lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml' 2024-07-01T14:47:45,394 adding 'lm_eval/tasks/mmlu/generative/mmlu_virology.yaml' 2024-07-01T14:47:45,395 adding 'lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml' 2024-07-01T14:47:45,398 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py' 2024-07-01T14:47:45,399 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml' 2024-07-01T14:47:45,400 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml' 2024-07-01T14:47:45,401 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml' 2024-07-01T14:47:45,403 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml' 2024-07-01T14:47:45,404 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml' 2024-07-01T14:47:45,405 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml' 2024-07-01T14:47:45,406 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml' 2024-07-01T14:47:45,407 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml' 2024-07-01T14:47:45,408 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml' 2024-07-01T14:47:45,410 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml' 2024-07-01T14:47:45,411 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml' 2024-07-01T14:47:45,412 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml' 2024-07-01T14:47:45,413 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml' 2024-07-01T14:47:45,414 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml' 2024-07-01T14:47:45,415 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml' 2024-07-01T14:47:45,416 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml' 2024-07-01T14:47:45,417 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml' 2024-07-01T14:47:45,418 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml' 2024-07-01T14:47:45,419 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml' 2024-07-01T14:47:45,420 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml' 2024-07-01T14:47:45,422 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml' 2024-07-01T14:47:45,423 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml' 2024-07-01T14:47:45,424 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml' 2024-07-01T14:47:45,425 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml' 2024-07-01T14:47:45,426 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml' 2024-07-01T14:47:45,427 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml' 2024-07-01T14:47:45,428 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml' 2024-07-01T14:47:45,429 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml' 2024-07-01T14:47:45,431 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml' 2024-07-01T14:47:45,432 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml' 2024-07-01T14:47:45,433 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml' 2024-07-01T14:47:45,434 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml' 2024-07-01T14:47:45,435 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml' 2024-07-01T14:47:45,436 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml' 2024-07-01T14:47:45,438 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml' 2024-07-01T14:47:45,439 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml' 2024-07-01T14:47:45,440 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml' 2024-07-01T14:47:45,441 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml' 2024-07-01T14:47:45,442 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml' 2024-07-01T14:47:45,443 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml' 2024-07-01T14:47:45,445 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml' 2024-07-01T14:47:45,446 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml' 2024-07-01T14:47:45,447 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml' 2024-07-01T14:47:45,448 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml' 2024-07-01T14:47:45,449 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml' 2024-07-01T14:47:45,451 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml' 2024-07-01T14:47:45,452 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml' 2024-07-01T14:47:45,453 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml' 2024-07-01T14:47:45,454 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml' 2024-07-01T14:47:45,455 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml' 2024-07-01T14:47:45,459 adding 'lm_eval/tasks/model_written_evals/persona/_generate_configs.py' 2024-07-01T14:47:45,460 adding 'lm_eval/tasks/model_written_evals/persona/_template_yaml' 2024-07-01T14:47:45,461 adding 'lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml' 2024-07-01T14:47:45,462 adding 'lm_eval/tasks/model_written_evals/persona/agreeableness.yaml' 2024-07-01T14:47:45,463 adding 'lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml' 2024-07-01T14:47:45,464 adding 'lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml' 2024-07-01T14:47:45,465 adding 'lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml' 2024-07-01T14:47:45,466 adding 'lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml' 2024-07-01T14:47:45,467 adding 'lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml' 2024-07-01T14:47:45,468 adding 'lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml' 2024-07-01T14:47:45,470 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml' 2024-07-01T14:47:45,471 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml' 2024-07-01T14:47:45,472 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml' 2024-07-01T14:47:45,473 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml' 2024-07-01T14:47:45,474 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml' 2024-07-01T14:47:45,475 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml' 2024-07-01T14:47:45,477 adding 'lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml' 2024-07-01T14:47:45,478 adding 'lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml' 2024-07-01T14:47:45,479 adding 'lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml' 2024-07-01T14:47:45,480 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml' 2024-07-01T14:47:45,481 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml' 2024-07-01T14:47:45,482 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml' 2024-07-01T14:47:45,483 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml' 2024-07-01T14:47:45,484 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml' 2024-07-01T14:47:45,486 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml' 2024-07-01T14:47:45,487 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml' 2024-07-01T14:47:45,488 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml' 2024-07-01T14:47:45,489 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml' 2024-07-01T14:47:45,490 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml' 2024-07-01T14:47:45,491 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml' 2024-07-01T14:47:45,493 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml' 2024-07-01T14:47:45,494 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml' 2024-07-01T14:47:45,495 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml' 2024-07-01T14:47:45,496 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml' 2024-07-01T14:47:45,497 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml' 2024-07-01T14:47:45,498 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml' 2024-07-01T14:47:45,499 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml' 2024-07-01T14:47:45,500 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml' 2024-07-01T14:47:45,501 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml' 2024-07-01T14:47:45,502 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml' 2024-07-01T14:47:45,503 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml' 2024-07-01T14:47:45,504 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml' 2024-07-01T14:47:45,506 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml' 2024-07-01T14:47:45,507 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml' 2024-07-01T14:47:45,508 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml' 2024-07-01T14:47:45,509 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml' 2024-07-01T14:47:45,510 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml' 2024-07-01T14:47:45,511 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml' 2024-07-01T14:47:45,512 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml' 2024-07-01T14:47:45,514 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml' 2024-07-01T14:47:45,515 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml' 2024-07-01T14:47:45,516 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml' 2024-07-01T14:47:45,517 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml' 2024-07-01T14:47:45,518 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml' 2024-07-01T14:47:45,519 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml' 2024-07-01T14:47:45,520 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml' 2024-07-01T14:47:45,521 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml' 2024-07-01T14:47:45,523 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml' 2024-07-01T14:47:45,524 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml' 2024-07-01T14:47:45,525 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml' 2024-07-01T14:47:45,526 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml' 2024-07-01T14:47:45,527 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml' 2024-07-01T14:47:45,528 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml' 2024-07-01T14:47:45,529 adding 'lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml' 2024-07-01T14:47:45,530 adding 'lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml' 2024-07-01T14:47:45,531 adding 'lm_eval/tasks/model_written_evals/persona/extraversion.yaml' 2024-07-01T14:47:45,533 adding 'lm_eval/tasks/model_written_evals/persona/has-disability.yaml' 2024-07-01T14:47:45,534 adding 'lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml' 2024-07-01T14:47:45,535 adding 'lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml' 2024-07-01T14:47:45,536 adding 'lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml' 2024-07-01T14:47:45,537 adding 'lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml' 2024-07-01T14:47:45,538 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml' 2024-07-01T14:47:45,539 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml' 2024-07-01T14:47:45,540 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml' 2024-07-01T14:47:45,542 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml' 2024-07-01T14:47:45,543 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml' 2024-07-01T14:47:45,544 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml' 2024-07-01T14:47:45,545 adding 'lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml' 2024-07-01T14:47:45,546 adding 'lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml' 2024-07-01T14:47:45,547 adding 'lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml' 2024-07-01T14:47:45,548 adding 'lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml' 2024-07-01T14:47:45,549 adding 'lm_eval/tasks/model_written_evals/persona/narcissism.yaml' 2024-07-01T14:47:45,551 adding 'lm_eval/tasks/model_written_evals/persona/neuroticism.yaml' 2024-07-01T14:47:45,552 adding 'lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml' 2024-07-01T14:47:45,553 adding 'lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml' 2024-07-01T14:47:45,554 adding 'lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml' 2024-07-01T14:47:45,555 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml' 2024-07-01T14:47:45,556 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml' 2024-07-01T14:47:45,557 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml' 2024-07-01T14:47:45,559 adding 'lm_eval/tasks/model_written_evals/persona/openness.yaml' 2024-07-01T14:47:45,560 adding 'lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml' 2024-07-01T14:47:45,561 adding 'lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml' 2024-07-01T14:47:45,562 adding 'lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml' 2024-07-01T14:47:45,563 adding 'lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml' 2024-07-01T14:47:45,564 adding 'lm_eval/tasks/model_written_evals/persona/psychopathy.yaml' 2024-07-01T14:47:45,565 adding 'lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml' 2024-07-01T14:47:45,567 adding 'lm_eval/tasks/model_written_evals/persona/risk-averse.yaml' 2024-07-01T14:47:45,568 adding 'lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml' 2024-07-01T14:47:45,569 adding 'lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml' 2024-07-01T14:47:45,570 adding 'lm_eval/tasks/model_written_evals/persona/self-replication.yaml' 2024-07-01T14:47:45,571 adding 'lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml' 2024-07-01T14:47:45,572 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml' 2024-07-01T14:47:45,574 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml' 2024-07-01T14:47:45,575 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml' 2024-07-01T14:47:45,576 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml' 2024-07-01T14:47:45,577 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml' 2024-07-01T14:47:45,578 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml' 2024-07-01T14:47:45,579 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml' 2024-07-01T14:47:45,580 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml' 2024-07-01T14:47:45,581 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml' 2024-07-01T14:47:45,583 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml' 2024-07-01T14:47:45,584 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml' 2024-07-01T14:47:45,585 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml' 2024-07-01T14:47:45,586 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml' 2024-07-01T14:47:45,587 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml' 2024-07-01T14:47:45,588 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml' 2024-07-01T14:47:45,589 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml' 2024-07-01T14:47:45,590 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml' 2024-07-01T14:47:45,591 adding 'lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml' 2024-07-01T14:47:45,592 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml' 2024-07-01T14:47:45,593 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml' 2024-07-01T14:47:45,595 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml' 2024-07-01T14:47:45,596 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml' 2024-07-01T14:47:45,597 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml' 2024-07-01T14:47:45,598 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml' 2024-07-01T14:47:45,599 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml' 2024-07-01T14:47:45,600 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml' 2024-07-01T14:47:45,601 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml' 2024-07-01T14:47:45,603 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml' 2024-07-01T14:47:45,604 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml' 2024-07-01T14:47:45,605 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml' 2024-07-01T14:47:45,606 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml' 2024-07-01T14:47:45,607 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml' 2024-07-01T14:47:45,608 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml' 2024-07-01T14:47:45,609 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml' 2024-07-01T14:47:45,611 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml' 2024-07-01T14:47:45,612 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml' 2024-07-01T14:47:45,613 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml' 2024-07-01T14:47:45,614 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml' 2024-07-01T14:47:45,616 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml' 2024-07-01T14:47:45,617 adding 'lm_eval/tasks/model_written_evals/winogenerated/_template_yaml' 2024-07-01T14:47:45,619 adding 'lm_eval/tasks/mutual/README.md' 2024-07-01T14:47:45,620 adding 'lm_eval/tasks/mutual/multual_plus.yaml' 2024-07-01T14:47:45,621 adding 'lm_eval/tasks/mutual/mutual.yaml' 2024-07-01T14:47:45,622 adding 'lm_eval/tasks/mutual/utils.py' 2024-07-01T14:47:45,624 adding 'lm_eval/tasks/noticia/README.md' 2024-07-01T14:47:45,625 adding 'lm_eval/tasks/noticia/noticia.yaml' 2024-07-01T14:47:45,626 adding 'lm_eval/tasks/noticia/utils.py' 2024-07-01T14:47:45,628 adding 'lm_eval/tasks/nq_open/README.md' 2024-07-01T14:47:45,629 adding 'lm_eval/tasks/nq_open/nq_open.yaml' 2024-07-01T14:47:45,632 adding 'lm_eval/tasks/okapi/arc_multilingual/README.md' 2024-07-01T14:47:45,633 adding 'lm_eval/tasks/okapi/arc_multilingual/_arc_yaml' 2024-07-01T14:47:45,634 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml' 2024-07-01T14:47:45,635 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml' 2024-07-01T14:47:45,636 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml' 2024-07-01T14:47:45,638 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml' 2024-07-01T14:47:45,639 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml' 2024-07-01T14:47:45,640 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml' 2024-07-01T14:47:45,641 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml' 2024-07-01T14:47:45,642 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml' 2024-07-01T14:47:45,643 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml' 2024-07-01T14:47:45,644 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml' 2024-07-01T14:47:45,645 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml' 2024-07-01T14:47:45,647 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml' 2024-07-01T14:47:45,648 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml' 2024-07-01T14:47:45,649 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml' 2024-07-01T14:47:45,650 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml' 2024-07-01T14:47:45,651 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml' 2024-07-01T14:47:45,652 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml' 2024-07-01T14:47:45,654 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml' 2024-07-01T14:47:45,655 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml' 2024-07-01T14:47:45,656 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml' 2024-07-01T14:47:45,657 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml' 2024-07-01T14:47:45,658 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml' 2024-07-01T14:47:45,659 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml' 2024-07-01T14:47:45,660 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml' 2024-07-01T14:47:45,662 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml' 2024-07-01T14:47:45,663 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml' 2024-07-01T14:47:45,664 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml' 2024-07-01T14:47:45,665 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml' 2024-07-01T14:47:45,666 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml' 2024-07-01T14:47:45,667 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml' 2024-07-01T14:47:45,669 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml' 2024-07-01T14:47:45,670 adding 'lm_eval/tasks/okapi/arc_multilingual/utils.py' 2024-07-01T14:47:45,672 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/README.md' 2024-07-01T14:47:45,673 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml' 2024-07-01T14:47:45,675 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml' 2024-07-01T14:47:45,676 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml' 2024-07-01T14:47:45,677 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml' 2024-07-01T14:47:45,678 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml' 2024-07-01T14:47:45,679 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml' 2024-07-01T14:47:45,680 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml' 2024-07-01T14:47:45,682 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml' 2024-07-01T14:47:45,683 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml' 2024-07-01T14:47:45,684 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml' 2024-07-01T14:47:45,685 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml' 2024-07-01T14:47:45,686 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml' 2024-07-01T14:47:45,687 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml' 2024-07-01T14:47:45,688 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml' 2024-07-01T14:47:45,689 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml' 2024-07-01T14:47:45,690 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml' 2024-07-01T14:47:45,691 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml' 2024-07-01T14:47:45,692 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml' 2024-07-01T14:47:45,694 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml' 2024-07-01T14:47:45,695 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml' 2024-07-01T14:47:45,696 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml' 2024-07-01T14:47:45,697 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml' 2024-07-01T14:47:45,698 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml' 2024-07-01T14:47:45,699 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml' 2024-07-01T14:47:45,700 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml' 2024-07-01T14:47:45,701 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml' 2024-07-01T14:47:45,703 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml' 2024-07-01T14:47:45,704 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml' 2024-07-01T14:47:45,705 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml' 2024-07-01T14:47:45,706 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml' 2024-07-01T14:47:45,707 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml' 2024-07-01T14:47:45,708 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/utils.py' 2024-07-01T14:47:45,711 adding 'lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml' 2024-07-01T14:47:45,712 adding 'lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py' 2024-07-01T14:47:45,713 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml' 2024-07-01T14:47:45,714 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml' 2024-07-01T14:47:45,715 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml' 2024-07-01T14:47:45,716 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml' 2024-07-01T14:47:45,717 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml' 2024-07-01T14:47:45,719 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml' 2024-07-01T14:47:45,720 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml' 2024-07-01T14:47:45,721 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml' 2024-07-01T14:47:45,722 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml' 2024-07-01T14:47:45,723 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml' 2024-07-01T14:47:45,724 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml' 2024-07-01T14:47:45,725 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml' 2024-07-01T14:47:45,726 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml' 2024-07-01T14:47:45,727 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml' 2024-07-01T14:47:45,728 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml' 2024-07-01T14:47:45,730 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml' 2024-07-01T14:47:45,731 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml' 2024-07-01T14:47:45,732 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml' 2024-07-01T14:47:45,733 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml' 2024-07-01T14:47:45,734 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml' 2024-07-01T14:47:45,735 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml' 2024-07-01T14:47:45,736 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml' 2024-07-01T14:47:45,737 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml' 2024-07-01T14:47:45,738 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml' 2024-07-01T14:47:45,739 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml' 2024-07-01T14:47:45,740 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml' 2024-07-01T14:47:45,742 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml' 2024-07-01T14:47:45,743 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml' 2024-07-01T14:47:45,744 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml' 2024-07-01T14:47:45,745 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml' 2024-07-01T14:47:45,746 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml' 2024-07-01T14:47:45,747 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml' 2024-07-01T14:47:45,748 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml' 2024-07-01T14:47:45,749 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml' 2024-07-01T14:47:45,752 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/README.md' 2024-07-01T14:47:45,753 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml' 2024-07-01T14:47:45,755 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml' 2024-07-01T14:47:45,756 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml' 2024-07-01T14:47:45,757 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml' 2024-07-01T14:47:45,758 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml' 2024-07-01T14:47:45,759 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml' 2024-07-01T14:47:45,760 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml' 2024-07-01T14:47:45,762 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml' 2024-07-01T14:47:45,763 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml' 2024-07-01T14:47:45,764 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml' 2024-07-01T14:47:45,765 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml' 2024-07-01T14:47:45,766 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml' 2024-07-01T14:47:45,767 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml' 2024-07-01T14:47:45,769 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml' 2024-07-01T14:47:45,770 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml' 2024-07-01T14:47:45,771 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml' 2024-07-01T14:47:45,772 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml' 2024-07-01T14:47:45,773 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml' 2024-07-01T14:47:45,774 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml' 2024-07-01T14:47:45,775 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml' 2024-07-01T14:47:45,776 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml' 2024-07-01T14:47:45,777 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml' 2024-07-01T14:47:45,779 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml' 2024-07-01T14:47:45,780 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml' 2024-07-01T14:47:45,781 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml' 2024-07-01T14:47:45,782 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml' 2024-07-01T14:47:45,783 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml' 2024-07-01T14:47:45,784 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml' 2024-07-01T14:47:45,785 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml' 2024-07-01T14:47:45,786 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml' 2024-07-01T14:47:45,787 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml' 2024-07-01T14:47:45,788 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml' 2024-07-01T14:47:45,789 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml' 2024-07-01T14:47:45,790 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml' 2024-07-01T14:47:45,791 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml' 2024-07-01T14:47:45,793 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml' 2024-07-01T14:47:45,794 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml' 2024-07-01T14:47:45,795 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml' 2024-07-01T14:47:45,796 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml' 2024-07-01T14:47:45,797 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml' 2024-07-01T14:47:45,798 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml' 2024-07-01T14:47:45,799 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml' 2024-07-01T14:47:45,801 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml' 2024-07-01T14:47:45,802 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml' 2024-07-01T14:47:45,803 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml' 2024-07-01T14:47:45,804 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml' 2024-07-01T14:47:45,805 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml' 2024-07-01T14:47:45,807 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml' 2024-07-01T14:47:45,808 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml' 2024-07-01T14:47:45,809 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml' 2024-07-01T14:47:45,810 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml' 2024-07-01T14:47:45,811 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml' 2024-07-01T14:47:45,812 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml' 2024-07-01T14:47:45,814 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml' 2024-07-01T14:47:45,815 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml' 2024-07-01T14:47:45,816 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml' 2024-07-01T14:47:45,817 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml' 2024-07-01T14:47:45,818 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml' 2024-07-01T14:47:45,819 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml' 2024-07-01T14:47:45,820 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml' 2024-07-01T14:47:45,821 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml' 2024-07-01T14:47:45,822 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml' 2024-07-01T14:47:45,823 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml' 2024-07-01T14:47:45,825 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml' 2024-07-01T14:47:45,826 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py' 2024-07-01T14:47:45,828 adding 'lm_eval/tasks/openbookqa/README.md' 2024-07-01T14:47:45,829 adding 'lm_eval/tasks/openbookqa/openbookqa.yaml' 2024-07-01T14:47:45,831 adding 'lm_eval/tasks/paloma/README.md' 2024-07-01T14:47:45,832 adding 'lm_eval/tasks/paloma/_paloma_template' 2024-07-01T14:47:45,833 adding 'lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml' 2024-07-01T14:47:45,834 adding 'lm_eval/tasks/paloma/paloma_c4_100_domains.yaml' 2024-07-01T14:47:45,835 adding 'lm_eval/tasks/paloma/paloma_c4_en.yaml' 2024-07-01T14:47:45,836 adding 'lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml' 2024-07-01T14:47:45,837 adding 'lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml' 2024-07-01T14:47:45,838 adding 'lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml' 2024-07-01T14:47:45,839 adding 'lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml' 2024-07-01T14:47:45,840 adding 'lm_eval/tasks/paloma/paloma_gab.yaml' 2024-07-01T14:47:45,841 adding 'lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml' 2024-07-01T14:47:45,843 adding 'lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml' 2024-07-01T14:47:45,844 adding 'lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml' 2024-07-01T14:47:45,845 adding 'lm_eval/tasks/paloma/paloma_mc4.yaml' 2024-07-01T14:47:45,846 adding 'lm_eval/tasks/paloma/paloma_ptb.yaml' 2024-07-01T14:47:45,847 adding 'lm_eval/tasks/paloma/paloma_redpajama.yaml' 2024-07-01T14:47:45,848 adding 'lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml' 2024-07-01T14:47:45,849 adding 'lm_eval/tasks/paloma/paloma_utils.py' 2024-07-01T14:47:45,850 adding 'lm_eval/tasks/paloma/paloma_wikitext_103.yaml' 2024-07-01T14:47:45,852 adding 'lm_eval/tasks/paws-x/README.md' 2024-07-01T14:47:45,854 adding 'lm_eval/tasks/paws-x/_generate_config.py' 2024-07-01T14:47:45,855 adding 'lm_eval/tasks/paws-x/paws_de.yaml' 2024-07-01T14:47:45,856 adding 'lm_eval/tasks/paws-x/paws_en.yaml' 2024-07-01T14:47:45,857 adding 'lm_eval/tasks/paws-x/paws_es.yaml' 2024-07-01T14:47:45,858 adding 'lm_eval/tasks/paws-x/paws_fr.yaml' 2024-07-01T14:47:45,859 adding 'lm_eval/tasks/paws-x/paws_ja.yaml' 2024-07-01T14:47:45,861 adding 'lm_eval/tasks/paws-x/paws_ko.yaml' 2024-07-01T14:47:45,862 adding 'lm_eval/tasks/paws-x/paws_zh.yaml' 2024-07-01T14:47:45,863 adding 'lm_eval/tasks/paws-x/pawsx_template_yaml' 2024-07-01T14:47:45,865 adding 'lm_eval/tasks/pile/README.md' 2024-07-01T14:47:45,866 adding 'lm_eval/tasks/pile/pile_arxiv.yaml' 2024-07-01T14:47:45,867 adding 'lm_eval/tasks/pile/pile_bookcorpus2.yaml' 2024-07-01T14:47:45,868 adding 'lm_eval/tasks/pile/pile_books3.yaml' 2024-07-01T14:47:45,870 adding 'lm_eval/tasks/pile/pile_dm-mathematics.yaml' 2024-07-01T14:47:45,871 adding 'lm_eval/tasks/pile/pile_enron.yaml' 2024-07-01T14:47:45,872 adding 'lm_eval/tasks/pile/pile_europarl.yaml' 2024-07-01T14:47:45,873 adding 'lm_eval/tasks/pile/pile_freelaw.yaml' 2024-07-01T14:47:45,874 adding 'lm_eval/tasks/pile/pile_github.yaml' 2024-07-01T14:47:45,875 adding 'lm_eval/tasks/pile/pile_gutenberg.yaml' 2024-07-01T14:47:45,876 adding 'lm_eval/tasks/pile/pile_hackernews.yaml' 2024-07-01T14:47:45,878 adding 'lm_eval/tasks/pile/pile_nih-exporter.yaml' 2024-07-01T14:47:45,879 adding 'lm_eval/tasks/pile/pile_opensubtitles.yaml' 2024-07-01T14:47:45,880 adding 'lm_eval/tasks/pile/pile_openwebtext2.yaml' 2024-07-01T14:47:45,881 adding 'lm_eval/tasks/pile/pile_philpapers.yaml' 2024-07-01T14:47:45,882 adding 'lm_eval/tasks/pile/pile_pile-cc.yaml' 2024-07-01T14:47:45,883 adding 'lm_eval/tasks/pile/pile_pubmed-abstracts.yaml' 2024-07-01T14:47:45,884 adding 'lm_eval/tasks/pile/pile_pubmed-central.yaml' 2024-07-01T14:47:45,885 adding 'lm_eval/tasks/pile/pile_stackexchange.yaml' 2024-07-01T14:47:45,886 adding 'lm_eval/tasks/pile/pile_ubuntu-irc.yaml' 2024-07-01T14:47:45,887 adding 'lm_eval/tasks/pile/pile_uspto.yaml' 2024-07-01T14:47:45,888 adding 'lm_eval/tasks/pile/pile_wikipedia.yaml' 2024-07-01T14:47:45,890 adding 'lm_eval/tasks/pile/pile_youtubesubtitles.yaml' 2024-07-01T14:47:45,891 adding 'lm_eval/tasks/pile_10k/README.md' 2024-07-01T14:47:45,893 adding 'lm_eval/tasks/pile_10k/pile_10k.yaml' 2024-07-01T14:47:45,894 adding 'lm_eval/tasks/piqa/README.md' 2024-07-01T14:47:45,896 adding 'lm_eval/tasks/piqa/piqa.yaml' 2024-07-01T14:47:45,897 adding 'lm_eval/tasks/polemo2/README.md' 2024-07-01T14:47:45,899 adding 'lm_eval/tasks/polemo2/polemo2_in.yaml' 2024-07-01T14:47:45,900 adding 'lm_eval/tasks/polemo2/polemo2_out.yaml' 2024-07-01T14:47:45,902 adding 'lm_eval/tasks/prost/README.md' 2024-07-01T14:47:45,903 adding 'lm_eval/tasks/prost/corypaik_prost.yaml' 2024-07-01T14:47:45,904 adding 'lm_eval/tasks/pubmedqa/README.md' 2024-07-01T14:47:45,906 adding 'lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py' 2024-07-01T14:47:45,907 adding 'lm_eval/tasks/pubmedqa/pubmedqa.yaml' 2024-07-01T14:47:45,908 adding 'lm_eval/tasks/qa4mre/README.md' 2024-07-01T14:47:45,909 adding 'lm_eval/tasks/qa4mre/preprocess_qa4mre.py' 2024-07-01T14:47:45,911 adding 'lm_eval/tasks/qa4mre/qa4mre_2011.yaml' 2024-07-01T14:47:45,912 adding 'lm_eval/tasks/qa4mre/qa4mre_2012.yaml' 2024-07-01T14:47:45,913 adding 'lm_eval/tasks/qa4mre/qa4mre_2013.yaml' 2024-07-01T14:47:45,915 adding 'lm_eval/tasks/qasper/README.md' 2024-07-01T14:47:45,916 adding 'lm_eval/tasks/qasper/bool.yaml' 2024-07-01T14:47:45,917 adding 'lm_eval/tasks/qasper/freeform.yaml' 2024-07-01T14:47:45,918 adding 'lm_eval/tasks/qasper/metrics.py' 2024-07-01T14:47:45,919 adding 'lm_eval/tasks/qasper/utils.py' 2024-07-01T14:47:45,921 adding 'lm_eval/tasks/race/README.md' 2024-07-01T14:47:45,922 adding 'lm_eval/tasks/race/preprocess_race.py' 2024-07-01T14:47:45,924 adding 'lm_eval/tasks/race/race.yaml' 2024-07-01T14:47:45,926 adding 'lm_eval/tasks/realtoxicityprompts/metric.py' 2024-07-01T14:47:45,927 adding 'lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml' 2024-07-01T14:47:45,929 adding 'lm_eval/tasks/sciq/README.md' 2024-07-01T14:47:45,930 adding 'lm_eval/tasks/sciq/sciq.yaml' 2024-07-01T14:47:45,932 adding 'lm_eval/tasks/scrolls/README.md' 2024-07-01T14:47:45,933 adding 'lm_eval/tasks/scrolls/scrolls_contractnli.yaml' 2024-07-01T14:47:45,934 adding 'lm_eval/tasks/scrolls/scrolls_govreport.yaml' 2024-07-01T14:47:45,935 adding 'lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml' 2024-07-01T14:47:45,936 adding 'lm_eval/tasks/scrolls/scrolls_qasper.yaml' 2024-07-01T14:47:45,937 adding 'lm_eval/tasks/scrolls/scrolls_qmsum.yaml' 2024-07-01T14:47:45,939 adding 'lm_eval/tasks/scrolls/scrolls_quality.yaml' 2024-07-01T14:47:45,940 adding 'lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml' 2024-07-01T14:47:45,942 adding 'lm_eval/tasks/scrolls/task.py' 2024-07-01T14:47:45,944 adding 'lm_eval/tasks/siqa/README.md' 2024-07-01T14:47:45,945 adding 'lm_eval/tasks/siqa/siqa.yaml' 2024-07-01T14:47:45,947 adding 'lm_eval/tasks/squad_completion/README.md' 2024-07-01T14:47:45,948 adding 'lm_eval/tasks/squad_completion/squad_completion.yaml' 2024-07-01T14:47:45,949 adding 'lm_eval/tasks/squad_completion/task.py' 2024-07-01T14:47:45,951 adding 'lm_eval/tasks/squadv2/README.md' 2024-07-01T14:47:45,955 adding 'lm_eval/tasks/squadv2/squadv2.yaml' 2024-07-01T14:47:45,956 adding 'lm_eval/tasks/squadv2/task.py' 2024-07-01T14:47:45,958 adding 'lm_eval/tasks/storycloze/README.md' 2024-07-01T14:47:45,959 adding 'lm_eval/tasks/storycloze/storycloze_2016.yaml' 2024-07-01T14:47:45,960 adding 'lm_eval/tasks/storycloze/storycloze_2018.yaml' 2024-07-01T14:47:45,962 adding 'lm_eval/tasks/super_glue/README.md' 2024-07-01T14:47:45,964 adding 'lm_eval/tasks/super_glue/boolq/default.yaml' 2024-07-01T14:47:45,965 adding 'lm_eval/tasks/super_glue/boolq/seq2seq.yaml' 2024-07-01T14:47:45,966 adding 'lm_eval/tasks/super_glue/boolq/t5-prompt.yaml' 2024-07-01T14:47:45,968 adding 'lm_eval/tasks/super_glue/cb/aggregate.py' 2024-07-01T14:47:45,969 adding 'lm_eval/tasks/super_glue/cb/default.yaml' 2024-07-01T14:47:45,970 adding 'lm_eval/tasks/super_glue/cb/t5-prompt.yaml' 2024-07-01T14:47:45,971 adding 'lm_eval/tasks/super_glue/cb/t5_utils.py' 2024-07-01T14:47:45,973 adding 'lm_eval/tasks/super_glue/copa/default.yaml' 2024-07-01T14:47:45,974 adding 'lm_eval/tasks/super_glue/copa/t5-prompt.yaml' 2024-07-01T14:47:45,975 adding 'lm_eval/tasks/super_glue/copa/utils.py' 2024-07-01T14:47:45,976 adding 'lm_eval/tasks/super_glue/multirc/default.yaml' 2024-07-01T14:47:45,978 adding 'lm_eval/tasks/super_glue/multirc/t5-prompt.yaml' 2024-07-01T14:47:45,979 adding 'lm_eval/tasks/super_glue/multirc/t5_utils.py' 2024-07-01T14:47:45,980 adding 'lm_eval/tasks/super_glue/record/default.yaml' 2024-07-01T14:47:45,982 adding 'lm_eval/tasks/super_glue/record/t5-prompt.yaml' 2024-07-01T14:47:45,983 adding 'lm_eval/tasks/super_glue/record/t5_utils.py' 2024-07-01T14:47:45,984 adding 'lm_eval/tasks/super_glue/record/util.py' 2024-07-01T14:47:45,986 adding 'lm_eval/tasks/super_glue/rte/default.yaml' 2024-07-01T14:47:45,987 adding 'lm_eval/tasks/super_glue/rte/t5-prompt.yaml' 2024-07-01T14:47:45,989 adding 'lm_eval/tasks/super_glue/wic/default.yaml' 2024-07-01T14:47:45,990 adding 'lm_eval/tasks/super_glue/wic/t5-prompt.yaml' 2024-07-01T14:47:45,991 adding 'lm_eval/tasks/super_glue/wsc/default.yaml' 2024-07-01T14:47:45,993 adding 'lm_eval/tasks/super_glue/wsc/preprocess_wsc.py' 2024-07-01T14:47:45,994 adding 'lm_eval/tasks/super_glue/wsc/t5-prompt.yaml' 2024-07-01T14:47:45,995 adding 'lm_eval/tasks/super_glue/wsc/t5_utils.py' 2024-07-01T14:47:45,997 adding 'lm_eval/tasks/swag/README.md' 2024-07-01T14:47:45,998 adding 'lm_eval/tasks/swag/swag.yaml' 2024-07-01T14:47:46,000 adding 'lm_eval/tasks/swde/README.md' 2024-07-01T14:47:46,001 adding 'lm_eval/tasks/swde/swde.yaml' 2024-07-01T14:47:46,003 adding 'lm_eval/tasks/swde/task.py' 2024-07-01T14:47:46,005 adding 'lm_eval/tasks/tinyBenchmarks/README.md' 2024-07-01T14:47:46,006 adding 'lm_eval/tasks/tinyBenchmarks/agg_functions.py' 2024-07-01T14:47:46,007 adding 'lm_eval/tasks/tinyBenchmarks/tinyArc.yaml' 2024-07-01T14:47:46,008 adding 'lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml' 2024-07-01T14:47:46,010 adding 'lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml' 2024-07-01T14:47:46,011 adding 'lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml' 2024-07-01T14:47:46,012 adding 'lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml' 2024-07-01T14:47:46,013 adding 'lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml' 2024-07-01T14:47:46,014 adding 'lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml' 2024-07-01T14:47:46,016 adding 'lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml' 2024-07-01T14:47:46,017 adding 'lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py' 2024-07-01T14:47:46,018 adding 'lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py' 2024-07-01T14:47:46,020 adding 'lm_eval/tasks/tinyBenchmarks/utils_winogrande.py' 2024-07-01T14:47:46,021 adding 'lm_eval/tasks/tmmluplus/README.md' 2024-07-01T14:47:46,023 adding 'lm_eval/tasks/tmmluplus/subject.tsv' 2024-07-01T14:47:46,026 adding 'lm_eval/tasks/tmmluplus/default/_default_template_yaml' 2024-07-01T14:47:46,027 adding 'lm_eval/tasks/tmmluplus/default/_generate_configs.py' 2024-07-01T14:47:46,028 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus.yaml' 2024-07-01T14:47:46,030 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml' 2024-07-01T14:47:46,031 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml' 2024-07-01T14:47:46,032 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml' 2024-07-01T14:47:46,038 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml' 2024-07-01T14:47:46,039 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml' 2024-07-01T14:47:46,040 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml' 2024-07-01T14:47:46,042 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml' 2024-07-01T14:47:46,043 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml' 2024-07-01T14:47:46,044 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml' 2024-07-01T14:47:46,045 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml' 2024-07-01T14:47:46,046 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml' 2024-07-01T14:47:46,047 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml' 2024-07-01T14:47:46,048 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml' 2024-07-01T14:47:46,049 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml' 2024-07-01T14:47:46,051 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml' 2024-07-01T14:47:46,052 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml' 2024-07-01T14:47:46,053 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml' 2024-07-01T14:47:46,054 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml' 2024-07-01T14:47:46,055 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml' 2024-07-01T14:47:46,056 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml' 2024-07-01T14:47:46,058 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml' 2024-07-01T14:47:46,059 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml' 2024-07-01T14:47:46,060 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml' 2024-07-01T14:47:46,061 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml' 2024-07-01T14:47:46,062 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml' 2024-07-01T14:47:46,063 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml' 2024-07-01T14:47:46,065 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml' 2024-07-01T14:47:46,066 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml' 2024-07-01T14:47:46,067 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml' 2024-07-01T14:47:46,068 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml' 2024-07-01T14:47:46,069 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml' 2024-07-01T14:47:46,070 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml' 2024-07-01T14:47:46,071 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml' 2024-07-01T14:47:46,073 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml' 2024-07-01T14:47:46,074 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml' 2024-07-01T14:47:46,075 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml' 2024-07-01T14:47:46,076 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml' 2024-07-01T14:47:46,077 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml' 2024-07-01T14:47:46,079 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml' 2024-07-01T14:47:46,080 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml' 2024-07-01T14:47:46,081 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml' 2024-07-01T14:47:46,082 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml' 2024-07-01T14:47:46,084 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml' 2024-07-01T14:47:46,085 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml' 2024-07-01T14:47:46,086 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml' 2024-07-01T14:47:46,087 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml' 2024-07-01T14:47:46,088 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml' 2024-07-01T14:47:46,089 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml' 2024-07-01T14:47:46,090 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml' 2024-07-01T14:47:46,091 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml' 2024-07-01T14:47:46,092 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml' 2024-07-01T14:47:46,094 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml' 2024-07-01T14:47:46,095 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml' 2024-07-01T14:47:46,096 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml' 2024-07-01T14:47:46,097 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml' 2024-07-01T14:47:46,098 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml' 2024-07-01T14:47:46,100 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml' 2024-07-01T14:47:46,101 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml' 2024-07-01T14:47:46,102 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml' 2024-07-01T14:47:46,103 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml' 2024-07-01T14:47:46,104 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml' 2024-07-01T14:47:46,105 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml' 2024-07-01T14:47:46,107 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml' 2024-07-01T14:47:46,108 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml' 2024-07-01T14:47:46,109 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml' 2024-07-01T14:47:46,110 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml' 2024-07-01T14:47:46,111 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml' 2024-07-01T14:47:46,113 adding 'lm_eval/tasks/tmmluplus/default/utils.py' 2024-07-01T14:47:46,114 adding 'lm_eval/tasks/toxigen/README.md' 2024-07-01T14:47:46,116 adding 'lm_eval/tasks/toxigen/toxigen.yaml' 2024-07-01T14:47:46,117 adding 'lm_eval/tasks/toxigen/utils.py' 2024-07-01T14:47:46,118 adding 'lm_eval/tasks/translation/README.md' 2024-07-01T14:47:46,120 adding 'lm_eval/tasks/translation/iwslt2017_ar-en.yaml' 2024-07-01T14:47:46,121 adding 'lm_eval/tasks/translation/iwslt2017_en-ar.yaml' 2024-07-01T14:47:46,122 adding 'lm_eval/tasks/translation/utils.py' 2024-07-01T14:47:46,123 adding 'lm_eval/tasks/translation/wmt14_en-fr.yaml' 2024-07-01T14:47:46,125 adding 'lm_eval/tasks/translation/wmt14_fr-en.yaml' 2024-07-01T14:47:46,126 adding 'lm_eval/tasks/translation/wmt16_de-en.yaml' 2024-07-01T14:47:46,127 adding 'lm_eval/tasks/translation/wmt16_en-de.yaml' 2024-07-01T14:47:46,128 adding 'lm_eval/tasks/translation/wmt16_en-ro.yaml' 2024-07-01T14:47:46,129 adding 'lm_eval/tasks/translation/wmt16_ro-en.yaml' 2024-07-01T14:47:46,130 adding 'lm_eval/tasks/translation/wmt_common_yaml' 2024-07-01T14:47:46,132 adding 'lm_eval/tasks/triviaqa/README.md' 2024-07-01T14:47:46,133 adding 'lm_eval/tasks/triviaqa/default.yaml' 2024-07-01T14:47:46,135 adding 'lm_eval/tasks/truthfulqa/README.md' 2024-07-01T14:47:46,137 adding 'lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml' 2024-07-01T14:47:46,138 adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml' 2024-07-01T14:47:46,139 adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml' 2024-07-01T14:47:46,140 adding 'lm_eval/tasks/truthfulqa/utils.py' 2024-07-01T14:47:46,142 adding 'lm_eval/tasks/unitxt/20_newsgroups.yaml' 2024-07-01T14:47:46,144 adding 'lm_eval/tasks/unitxt/README.md' 2024-07-01T14:47:46,145 adding 'lm_eval/tasks/unitxt/ag_news.yaml' 2024-07-01T14:47:46,146 adding 'lm_eval/tasks/unitxt/argument_topic.yaml' 2024-07-01T14:47:46,147 adding 'lm_eval/tasks/unitxt/atis.yaml' 2024-07-01T14:47:46,148 adding 'lm_eval/tasks/unitxt/banking77.yaml' 2024-07-01T14:47:46,149 adding 'lm_eval/tasks/unitxt/claim_stance_topic.yaml' 2024-07-01T14:47:46,150 adding 'lm_eval/tasks/unitxt/cnn_dailymail.yaml' 2024-07-01T14:47:46,151 adding 'lm_eval/tasks/unitxt/coedit_gec.yaml' 2024-07-01T14:47:46,152 adding 'lm_eval/tasks/unitxt/dbpedia_14.yaml' 2024-07-01T14:47:46,153 adding 'lm_eval/tasks/unitxt/ethos_binary.yaml' 2024-07-01T14:47:46,154 adding 'lm_eval/tasks/unitxt/financial_tweets.yaml' 2024-07-01T14:47:46,156 adding 'lm_eval/tasks/unitxt/generate_yamls.py' 2024-07-01T14:47:46,157 adding 'lm_eval/tasks/unitxt/law_stack_exchange.yaml' 2024-07-01T14:47:46,158 adding 'lm_eval/tasks/unitxt/ledgar.yaml' 2024-07-01T14:47:46,159 adding 'lm_eval/tasks/unitxt/medical_abstracts.yaml' 2024-07-01T14:47:46,160 adding 'lm_eval/tasks/unitxt/stsb.yaml' 2024-07-01T14:47:46,162 adding 'lm_eval/tasks/unitxt/unfair_tos.yaml' 2024-07-01T14:47:46,163 adding 'lm_eval/tasks/unitxt/unitxt_datasets' 2024-07-01T14:47:46,164 adding 'lm_eval/tasks/unitxt/unitxt_tasks.classification.multi_class' 2024-07-01T14:47:46,165 adding 'lm_eval/tasks/unitxt/unitxt_tasks.classification.multi_label' 2024-07-01T14:47:46,166 adding 'lm_eval/tasks/unitxt/unitxt_tasks.grammatical_error_correction' 2024-07-01T14:47:46,167 adding 'lm_eval/tasks/unitxt/unitxt_tasks.qa.with_context.extractive' 2024-07-01T14:47:46,169 adding 'lm_eval/tasks/unitxt/unitxt_tasks.regression.two_texts' 2024-07-01T14:47:46,170 adding 'lm_eval/tasks/unitxt/unitxt_tasks.span_labeling.extraction' 2024-07-01T14:47:46,171 adding 'lm_eval/tasks/unitxt/unitxt_tasks.summarization.abstractive' 2024-07-01T14:47:46,172 adding 'lm_eval/tasks/unitxt/unitxt_wrapper.py' 2024-07-01T14:47:46,173 adding 'lm_eval/tasks/unitxt/xsum.yaml' 2024-07-01T14:47:46,174 adding 'lm_eval/tasks/unitxt/yahoo_answers_topics.yaml' 2024-07-01T14:47:46,176 adding 'lm_eval/tasks/unscramble/README.md' 2024-07-01T14:47:46,177 adding 'lm_eval/tasks/unscramble/anagrams1.yaml' 2024-07-01T14:47:46,178 adding 'lm_eval/tasks/unscramble/anagrams2.yaml' 2024-07-01T14:47:46,179 adding 'lm_eval/tasks/unscramble/cycle_letters.yaml' 2024-07-01T14:47:46,181 adding 'lm_eval/tasks/unscramble/random_insertion.yaml' 2024-07-01T14:47:46,182 adding 'lm_eval/tasks/unscramble/reversed_words.yaml' 2024-07-01T14:47:46,184 adding 'lm_eval/tasks/webqs/README.md' 2024-07-01T14:47:46,185 adding 'lm_eval/tasks/webqs/utils.py' 2024-07-01T14:47:46,186 adding 'lm_eval/tasks/webqs/webqs.yaml' 2024-07-01T14:47:46,188 adding 'lm_eval/tasks/wikitext/README.md' 2024-07-01T14:47:46,189 adding 'lm_eval/tasks/wikitext/preprocess_wikitext.py' 2024-07-01T14:47:46,190 adding 'lm_eval/tasks/wikitext/wikitext.yaml' 2024-07-01T14:47:46,192 adding 'lm_eval/tasks/winogrande/README.md' 2024-07-01T14:47:46,193 adding 'lm_eval/tasks/winogrande/default.yaml' 2024-07-01T14:47:46,194 adding 'lm_eval/tasks/winogrande/preprocess_winogrande.py' 2024-07-01T14:47:46,196 adding 'lm_eval/tasks/wmdp/README.md' 2024-07-01T14:47:46,197 adding 'lm_eval/tasks/wmdp/_default_template_yaml' 2024-07-01T14:47:46,199 adding 'lm_eval/tasks/wmdp/wmdp_bio.yaml' 2024-07-01T14:47:46,200 adding 'lm_eval/tasks/wmdp/wmdp_chem.yaml' 2024-07-01T14:47:46,201 adding 'lm_eval/tasks/wmdp/wmdp_cyber.yaml' 2024-07-01T14:47:46,203 adding 'lm_eval/tasks/wmt2016/README.md' 2024-07-01T14:47:46,204 adding 'lm_eval/tasks/wmt2016/metrics.py' 2024-07-01T14:47:46,205 adding 'lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml' 2024-07-01T14:47:46,207 adding 'lm_eval/tasks/wsc273/README.md' 2024-07-01T14:47:46,208 adding 'lm_eval/tasks/wsc273/default.yaml' 2024-07-01T14:47:46,209 adding 'lm_eval/tasks/wsc273/utils.py' 2024-07-01T14:47:46,211 adding 'lm_eval/tasks/xcopa/README.md' 2024-07-01T14:47:46,212 adding 'lm_eval/tasks/xcopa/default_et.yaml' 2024-07-01T14:47:46,213 adding 'lm_eval/tasks/xcopa/default_ht.yaml' 2024-07-01T14:47:46,214 adding 'lm_eval/tasks/xcopa/default_id.yaml' 2024-07-01T14:47:46,215 adding 'lm_eval/tasks/xcopa/default_it.yaml' 2024-07-01T14:47:46,216 adding 'lm_eval/tasks/xcopa/default_qu.yaml' 2024-07-01T14:47:46,218 adding 'lm_eval/tasks/xcopa/default_sw.yaml' 2024-07-01T14:47:46,219 adding 'lm_eval/tasks/xcopa/default_ta.yaml' 2024-07-01T14:47:46,220 adding 'lm_eval/tasks/xcopa/default_th.yaml' 2024-07-01T14:47:46,221 adding 'lm_eval/tasks/xcopa/default_tr.yaml' 2024-07-01T14:47:46,222 adding 'lm_eval/tasks/xcopa/default_vi.yaml' 2024-07-01T14:47:46,223 adding 'lm_eval/tasks/xcopa/default_zh.yaml' 2024-07-01T14:47:46,224 adding 'lm_eval/tasks/xcopa/utils.py' 2024-07-01T14:47:46,226 adding 'lm_eval/tasks/xnli/README.md' 2024-07-01T14:47:46,227 adding 'lm_eval/tasks/xnli/utils.py' 2024-07-01T14:47:46,228 adding 'lm_eval/tasks/xnli/xnli_ar.yaml' 2024-07-01T14:47:46,229 adding 'lm_eval/tasks/xnli/xnli_bg.yaml' 2024-07-01T14:47:46,230 adding 'lm_eval/tasks/xnli/xnli_common_yaml' 2024-07-01T14:47:46,232 adding 'lm_eval/tasks/xnli/xnli_de.yaml' 2024-07-01T14:47:46,233 adding 'lm_eval/tasks/xnli/xnli_el.yaml' 2024-07-01T14:47:46,234 adding 'lm_eval/tasks/xnli/xnli_en.yaml' 2024-07-01T14:47:46,235 adding 'lm_eval/tasks/xnli/xnli_es.yaml' 2024-07-01T14:47:46,236 adding 'lm_eval/tasks/xnli/xnli_fr.yaml' 2024-07-01T14:47:46,237 adding 'lm_eval/tasks/xnli/xnli_hi.yaml' 2024-07-01T14:47:46,238 adding 'lm_eval/tasks/xnli/xnli_ru.yaml' 2024-07-01T14:47:46,239 adding 'lm_eval/tasks/xnli/xnli_sw.yaml' 2024-07-01T14:47:46,241 adding 'lm_eval/tasks/xnli/xnli_th.yaml' 2024-07-01T14:47:46,242 adding 'lm_eval/tasks/xnli/xnli_tr.yaml' 2024-07-01T14:47:46,243 adding 'lm_eval/tasks/xnli/xnli_ur.yaml' 2024-07-01T14:47:46,244 adding 'lm_eval/tasks/xnli/xnli_vi.yaml' 2024-07-01T14:47:46,245 adding 'lm_eval/tasks/xnli/xnli_zh.yaml' 2024-07-01T14:47:46,247 adding 'lm_eval/tasks/xnli_eu/README.md' 2024-07-01T14:47:46,248 adding 'lm_eval/tasks/xnli_eu/xnli_common_yaml' 2024-07-01T14:47:46,250 adding 'lm_eval/tasks/xnli_eu/xnli_eu.yaml' 2024-07-01T14:47:46,251 adding 'lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml' 2024-07-01T14:47:46,252 adding 'lm_eval/tasks/xnli_eu/xnli_eu_native.yaml' 2024-07-01T14:47:46,254 adding 'lm_eval/tasks/xstorycloze/README.md' 2024-07-01T14:47:46,255 adding 'lm_eval/tasks/xstorycloze/default_ar.yaml' 2024-07-01T14:47:46,256 adding 'lm_eval/tasks/xstorycloze/default_en.yaml' 2024-07-01T14:47:46,257 adding 'lm_eval/tasks/xstorycloze/default_es.yaml' 2024-07-01T14:47:46,259 adding 'lm_eval/tasks/xstorycloze/default_eu.yaml' 2024-07-01T14:47:46,260 adding 'lm_eval/tasks/xstorycloze/default_hi.yaml' 2024-07-01T14:47:46,261 adding 'lm_eval/tasks/xstorycloze/default_id.yaml' 2024-07-01T14:47:46,262 adding 'lm_eval/tasks/xstorycloze/default_my.yaml' 2024-07-01T14:47:46,263 adding 'lm_eval/tasks/xstorycloze/default_ru.yaml' 2024-07-01T14:47:46,264 adding 'lm_eval/tasks/xstorycloze/default_sw.yaml' 2024-07-01T14:47:46,265 adding 'lm_eval/tasks/xstorycloze/default_te.yaml' 2024-07-01T14:47:46,267 adding 'lm_eval/tasks/xstorycloze/default_zh.yaml' 2024-07-01T14:47:46,269 adding 'lm_eval/tasks/xwinograd/README.md' 2024-07-01T14:47:46,270 adding 'lm_eval/tasks/xwinograd/utils.py' 2024-07-01T14:47:46,271 adding 'lm_eval/tasks/xwinograd/xwinograd_common_yaml' 2024-07-01T14:47:46,272 adding 'lm_eval/tasks/xwinograd/xwinograd_en.yaml' 2024-07-01T14:47:46,273 adding 'lm_eval/tasks/xwinograd/xwinograd_fr.yaml' 2024-07-01T14:47:46,275 adding 'lm_eval/tasks/xwinograd/xwinograd_jp.yaml' 2024-07-01T14:47:46,276 adding 'lm_eval/tasks/xwinograd/xwinograd_pt.yaml' 2024-07-01T14:47:46,277 adding 'lm_eval/tasks/xwinograd/xwinograd_ru.yaml' 2024-07-01T14:47:46,278 adding 'lm_eval/tasks/xwinograd/xwinograd_zh.yaml' 2024-07-01T14:47:46,280 adding 'lm_eval-0.4.3.dist-info/LICENSE.md' 2024-07-01T14:47:46,285 adding 'lm_eval-0.4.3.dist-info/METADATA' 2024-07-01T14:47:46,286 adding 'lm_eval-0.4.3.dist-info/WHEEL' 2024-07-01T14:47:46,287 adding 'lm_eval-0.4.3.dist-info/entry_points.txt' 2024-07-01T14:47:46,288 adding 'lm_eval-0.4.3.dist-info/top_level.txt' 2024-07-01T14:47:46,333 adding 'lm_eval-0.4.3.dist-info/RECORD' 2024-07-01T14:47:46,384 removing build/bdist.linux-armv7l/wheel 2024-07-01T14:47:47,051 Building wheel for lm-eval (pyproject.toml): finished with status 'done' 2024-07-01T14:47:47,100 Created wheel for lm-eval: filename=lm_eval-0.4.3-py3-none-any.whl size=1655094 sha256=7f4cb7cbd7ea20f2624f1315e69dfbc884cc187749c85760f9b4ec03e7efe491 2024-07-01T14:47:47,101 Stored in directory: /tmp/pip-ephem-wheel-cache-b85g5w9w/wheels/7d/15/21/2d5dc17596e5d79d20cd86f85beba30749c885c9391b7d364c 2024-07-01T14:47:47,198 Successfully built lm-eval 2024-07-01T14:47:47,240 Removed build tracker: '/tmp/pip-build-tracker-pqp17b_0'