2024-01-31T16:25:08,165 Created temporary directory: /tmp/pip-build-tracker-eq5cqtfn
2024-01-31T16:25:08,166 Initialized build tracking at /tmp/pip-build-tracker-eq5cqtfn
2024-01-31T16:25:08,167 Created build tracker: /tmp/pip-build-tracker-eq5cqtfn
2024-01-31T16:25:08,167 Entered build tracker: /tmp/pip-build-tracker-eq5cqtfn
2024-01-31T16:25:08,168 Created temporary directory: /tmp/pip-wheel-fqhmybac
2024-01-31T16:25:08,171 Created temporary directory: /tmp/pip-ephem-wheel-cache-ctj9ybzs
2024-01-31T16:25:08,193 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
2024-01-31T16:25:08,196 2 location(s) to search for versions of lm-eval:
2024-01-31T16:25:08,196 * https://pypi.org/simple/lm-eval/
2024-01-31T16:25:08,196 * https://www.piwheels.org/simple/lm-eval/
2024-01-31T16:25:08,197 Fetching project page and analyzing links: https://pypi.org/simple/lm-eval/
2024-01-31T16:25:08,198 Getting page https://pypi.org/simple/lm-eval/
2024-01-31T16:25:08,199 Found index url https://pypi.org/simple/
2024-01-31T16:25:08,419 Fetched page https://pypi.org/simple/lm-eval/ as application/vnd.pypi.simple.v1+json
2024-01-31T16:25:08,421   Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/1b/5f/7841febb99c12ffb453d33a67b9841e89dba18c388b644bf22b81d137fc4/lm_eval-0.0.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6)
2024-01-31T16:25:08,422   Found link https://files.pythonhosted.org/packages/21/5a/feb5ff3a1591ca963c54873d39116b0e6a4f80e493e961ac08569709c5d7/lm_eval-0.0.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.0.1
2024-01-31T16:25:08,423   Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/f3/a7/63cbce8b51de25fabb1c49f3a3fd1704faaacadb5ed816401f800e4d2dbd/lm_eval-0.2.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6)
2024-01-31T16:25:08,424   Found link https://files.pythonhosted.org/packages/c5/fd/edd21b0f258b4ec0260f99f5b2ac3864f7cddc8fb7c83bbb2379a6aab975/lm_eval-0.2.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.2.0
2024-01-31T16:25:08,424   Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/61/c5/bff92e6b61fc2b0c1b7ac769633731910152e5176a404912ce7c07329ba0/lm_eval-0.3.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6)
2024-01-31T16:25:08,425   Found link https://files.pythonhosted.org/packages/c4/f8/58abc65390a758c8c2e5f1d8bb9b58d7885d02535d5f48de27006453d07e/lm_eval-0.3.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.3.0
2024-01-31T16:25:08,426   Found link https://files.pythonhosted.org/packages/45/e0/05001c2e56e2f8f793189442176432ddb89a2f1f1ef00ea154d0bc00fe37/lm_eval-0.4.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.0
2024-01-31T16:25:08,427   Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/49/2d/39f7a25ab663cb45cfc7773b85980f01df44853cc427d00dce94c90b43e6/lm_eval-0.4.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8)
2024-01-31T16:25:08,428   Found link https://files.pythonhosted.org/packages/5a/02/1c7f1ac2f139f4c05af5b94e2c4f88a70404fa0b0c22a5fb04dec0216b03/lm_eval-0.4.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.1
2024-01-31T16:25:08,429 Fetching project page and analyzing links: https://www.piwheels.org/simple/lm-eval/
2024-01-31T16:25:08,429 Getting page https://www.piwheels.org/simple/lm-eval/
2024-01-31T16:25:08,431 Found index url https://www.piwheels.org/simple/
2024-01-31T16:25:08,599 Fetched page https://www.piwheels.org/simple/lm-eval/ as text/html
2024-01-31T16:25:08,602   Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.4.0-py3-none-any.whl#sha256=1a863b5f478f2e66e921dbbf2e4dcda06a923021763f01b59c5b03bdd3af01c2 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8)
2024-01-31T16:25:08,602   Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.3.0-py3-none-any.whl#sha256=498b8b8954c1f9c17f46e3ec096e9be6b9c96ee70560ee613a4eb9c7b9d31644 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6)
2024-01-31T16:25:08,603   Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.2.0-py3-none-any.whl#sha256=e06d3d7b6016be832e6889cbc9c4787b99156ba57f8feb31d3aeb27304c6558c (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6)
2024-01-31T16:25:08,603   Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.0.1-py3-none-any.whl#sha256=0afc289f69286f71017fb9811dfea6cda7c703bf693ad43106fbbff1f164cf14 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6)
2024-01-31T16:25:08,604 Skipping link: not a file: https://www.piwheels.org/simple/lm-eval/
2024-01-31T16:25:08,604 Skipping link: not a file: https://pypi.org/simple/lm-eval/
2024-01-31T16:25:08,624 Given no hashes to check 1 links for project 'lm-eval': discarding no candidates
2024-01-31T16:25:08,643 Collecting lm-eval==0.4.1
2024-01-31T16:25:08,646   Created temporary directory: /tmp/pip-unpack-ucvssvq3
2024-01-31T16:25:08,857   Downloading lm_eval-0.4.1.tar.gz (504 kB)
2024-01-31T16:25:11,472   Added lm-eval==0.4.1 from https://files.pythonhosted.org/packages/5a/02/1c7f1ac2f139f4c05af5b94e2c4f88a70404fa0b0c22a5fb04dec0216b03/lm_eval-0.4.1.tar.gz to build tracker '/tmp/pip-build-tracker-eq5cqtfn'
2024-01-31T16:25:11,479   Created temporary directory: /tmp/pip-build-env-q8nmpx6z
2024-01-31T16:25:11,483   Installing build dependencies: started
2024-01-31T16:25:11,484   Running command pip subprocess to install build dependencies
2024-01-31T16:25:12,675   Using pip 23.3.1 from /home/piwheels/.local/lib/python3.11/site-packages/pip (python 3.11)
2024-01-31T16:25:13,217   Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
2024-01-31T16:25:14,708   Collecting setuptools>=40.8.0
2024-01-31T16:25:14,725     Using cached https://www.piwheels.org/simple/setuptools/setuptools-69.0.3-py3-none-any.whl (819 kB)
2024-01-31T16:25:14,956   Collecting wheel
2024-01-31T16:25:14,971     Using cached https://www.piwheels.org/simple/wheel/wheel-0.42.0-py3-none-any.whl (65 kB)
2024-01-31T16:25:17,674   Installing collected packages: wheel, setuptools
2024-01-31T16:25:17,911     Creating /tmp/pip-build-env-q8nmpx6z/overlay/local/bin
2024-01-31T16:25:17,913     changing mode of /tmp/pip-build-env-q8nmpx6z/overlay/local/bin/wheel to 755
2024-01-31T16:25:20,481   Successfully installed setuptools-69.0.3 wheel-0.42.0

2024-01-31T16:25:20,743   [notice] A new release of pip is available: 23.3.1 -> 23.3.2
2024-01-31T16:25:20,744   [notice] To update, run: python3 -m pip install --upgrade pip
2024-01-31T16:25:21,036   Installing build dependencies: finished with status 'done'
2024-01-31T16:25:21,040   Getting requirements to build wheel: started
2024-01-31T16:25:21,041   Running command Getting requirements to build wheel
2024-01-31T16:25:21,929   running egg_info
2024-01-31T16:25:21,933   writing lm_eval.egg-info/PKG-INFO
2024-01-31T16:25:21,951   writing dependency_links to lm_eval.egg-info/dependency_links.txt
2024-01-31T16:25:21,953   writing entry points to lm_eval.egg-info/entry_points.txt
2024-01-31T16:25:21,962   writing requirements to lm_eval.egg-info/requires.txt
2024-01-31T16:25:21,963   writing top-level names to lm_eval.egg-info/top_level.txt
2024-01-31T16:25:22,372   reading manifest file 'lm_eval.egg-info/SOURCES.txt'
2024-01-31T16:25:22,410   adding license file 'LICENSE.md'
2024-01-31T16:25:22,493   writing manifest file 'lm_eval.egg-info/SOURCES.txt'
2024-01-31T16:25:22,614   Getting requirements to build wheel: finished with status 'done'
2024-01-31T16:25:22,624   Created temporary directory: /tmp/pip-modern-metadata-2c1jlz7c
2024-01-31T16:25:22,626   Preparing metadata (pyproject.toml): started
2024-01-31T16:25:22,627   Running command Preparing metadata (pyproject.toml)
2024-01-31T16:25:23,459   running dist_info
2024-01-31T16:25:23,463   creating /tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info
2024-01-31T16:25:23,467   writing /tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/PKG-INFO
2024-01-31T16:25:23,485   writing dependency_links to /tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/dependency_links.txt
2024-01-31T16:25:23,486   writing entry points to /tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/entry_points.txt
2024-01-31T16:25:23,495   writing requirements to /tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/requires.txt
2024-01-31T16:25:23,497   writing top-level names to /tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/top_level.txt
2024-01-31T16:25:23,498   writing manifest file '/tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/SOURCES.txt'
2024-01-31T16:25:23,890   reading manifest file '/tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/SOURCES.txt'
2024-01-31T16:25:23,892   adding license file 'LICENSE.md'
2024-01-31T16:25:23,950   writing manifest file '/tmp/pip-modern-metadata-2c1jlz7c/lm_eval.egg-info/SOURCES.txt'
2024-01-31T16:25:23,991   creating '/tmp/pip-modern-metadata-2c1jlz7c/lm_eval-0.4.1.dist-info'
2024-01-31T16:25:24,379   Preparing metadata (pyproject.toml): finished with status 'done'
2024-01-31T16:25:24,385   Source in /tmp/pip-wheel-fqhmybac/lm-eval_c760d6f72f1844b3b6a281a833afe29d has version 0.4.1, which satisfies requirement lm-eval==0.4.1 from https://files.pythonhosted.org/packages/5a/02/1c7f1ac2f139f4c05af5b94e2c4f88a70404fa0b0c22a5fb04dec0216b03/lm_eval-0.4.1.tar.gz
2024-01-31T16:25:24,386   Removed lm-eval==0.4.1 from https://files.pythonhosted.org/packages/5a/02/1c7f1ac2f139f4c05af5b94e2c4f88a70404fa0b0c22a5fb04dec0216b03/lm_eval-0.4.1.tar.gz from build tracker '/tmp/pip-build-tracker-eq5cqtfn'
2024-01-31T16:25:24,396 Created temporary directory: /tmp/pip-unpack-kqvv1l8k
2024-01-31T16:25:24,397 Created temporary directory: /tmp/pip-unpack-hfirjz85
2024-01-31T16:25:24,534 Building wheels for collected packages: lm-eval
2024-01-31T16:25:24,538   Created temporary directory: /tmp/pip-wheel-sd4il15g
2024-01-31T16:25:24,538   Destination directory: /tmp/pip-wheel-sd4il15g
2024-01-31T16:25:24,541   Building wheel for lm-eval (pyproject.toml): started
2024-01-31T16:25:24,542   Running command Building wheel for lm-eval (pyproject.toml)
2024-01-31T16:25:25,351   running bdist_wheel
2024-01-31T16:25:25,367   running build
2024-01-31T16:25:25,368   running build_py
2024-01-31T16:25:25,372   creating build
2024-01-31T16:25:25,372   creating build/lib
2024-01-31T16:25:25,373   creating build/lib/lm_eval
2024-01-31T16:25:25,374   copying lm_eval/__init__.py -> build/lib/lm_eval
2024-01-31T16:25:25,376   copying lm_eval/utils.py -> build/lib/lm_eval
2024-01-31T16:25:25,379   copying lm_eval/__main__.py -> build/lib/lm_eval
2024-01-31T16:25:25,381   copying lm_eval/evaluator.py -> build/lib/lm_eval
2024-01-31T16:25:25,384   creating build/lib/lm_eval/tasks
2024-01-31T16:25:25,385   copying lm_eval/tasks/__init__.py -> build/lib/lm_eval/tasks
2024-01-31T16:25:25,388   creating build/lib/lm_eval/prompts
2024-01-31T16:25:25,389   copying lm_eval/prompts/__init__.py -> build/lib/lm_eval/prompts
2024-01-31T16:25:25,391   creating build/lib/lm_eval/decontamination
2024-01-31T16:25:25,392   copying lm_eval/decontamination/decontaminate.py -> build/lib/lm_eval/decontamination
2024-01-31T16:25:25,395   copying lm_eval/decontamination/__init__.py -> build/lib/lm_eval/decontamination
2024-01-31T16:25:25,396   copying lm_eval/decontamination/janitor.py -> build/lib/lm_eval/decontamination
2024-01-31T16:25:25,398   copying lm_eval/decontamination/archiver.py -> build/lib/lm_eval/decontamination
2024-01-31T16:25:25,401   creating build/lib/lm_eval/filters
2024-01-31T16:25:25,402   copying lm_eval/filters/decontamination.py -> build/lib/lm_eval/filters
2024-01-31T16:25:25,404   copying lm_eval/filters/extraction.py -> build/lib/lm_eval/filters
2024-01-31T16:25:25,405   copying lm_eval/filters/__init__.py -> build/lib/lm_eval/filters
2024-01-31T16:25:25,407   copying lm_eval/filters/transformation.py -> build/lib/lm_eval/filters
2024-01-31T16:25:25,409   copying lm_eval/filters/selection.py -> build/lib/lm_eval/filters
2024-01-31T16:25:25,411   creating build/lib/lm_eval/models
2024-01-31T16:25:25,412   copying lm_eval/models/huggingface.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,415   copying lm_eval/models/__init__.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,417   copying lm_eval/models/optimum_lm.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,419   copying lm_eval/models/mamba_lm.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,421   copying lm_eval/models/gguf.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,423   copying lm_eval/models/anthropic_llms.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,425   copying lm_eval/models/openai_completions.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,427   copying lm_eval/models/vllm_causallms.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,430   copying lm_eval/models/textsynth.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,432   copying lm_eval/models/dummy.py -> build/lib/lm_eval/models
2024-01-31T16:25:25,434   creating build/lib/lm_eval/api
2024-01-31T16:25:25,435   copying lm_eval/api/model.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,437   copying lm_eval/api/metrics.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,439   copying lm_eval/api/__init__.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,441   copying lm_eval/api/samplers.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,442   copying lm_eval/api/instance.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,444   copying lm_eval/api/task.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,447   copying lm_eval/api/registry.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,449   copying lm_eval/api/filter.py -> build/lib/lm_eval/api
2024-01-31T16:25:25,451   creating build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:25,452   copying lm_eval/tasks/hendrycks_ethics/utils.py -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:25,454   creating build/lib/lm_eval/tasks/medmcqa
2024-01-31T16:25:25,455   copying lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/lib/lm_eval/tasks/medmcqa
2024-01-31T16:25:25,458   creating build/lib/lm_eval/tasks/truthfulqa
2024-01-31T16:25:25,459   copying lm_eval/tasks/truthfulqa/utils.py -> build/lib/lm_eval/tasks/truthfulqa
2024-01-31T16:25:25,462   creating build/lib/lm_eval/tasks/mutual
2024-01-31T16:25:25,463   copying lm_eval/tasks/mutual/utils.py -> build/lib/lm_eval/tasks/mutual
2024-01-31T16:25:25,465   creating build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:25,467   copying lm_eval/tasks/ceval/_generate_configs.py -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:25,469   creating build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:25,470   copying lm_eval/tasks/crows_pairs/utils.py -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:25,473   creating build/lib/lm_eval/tasks/ifeval
2024-01-31T16:25:25,473   copying lm_eval/tasks/ifeval/utils.py -> build/lib/lm_eval/tasks/ifeval
2024-01-31T16:25:25,476   copying lm_eval/tasks/ifeval/instructions_registry.py -> build/lib/lm_eval/tasks/ifeval
2024-01-31T16:25:25,478   copying lm_eval/tasks/ifeval/instructions.py -> build/lib/lm_eval/tasks/ifeval
2024-01-31T16:25:25,481   copying lm_eval/tasks/ifeval/instructions_util.py -> build/lib/lm_eval/tasks/ifeval
2024-01-31T16:25:25,485   creating build/lib/lm_eval/tasks/pubmedqa
2024-01-31T16:25:25,486   copying lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/lib/lm_eval/tasks/pubmedqa
2024-01-31T16:25:25,488   creating build/lib/lm_eval/tasks/wmt2016
2024-01-31T16:25:25,489   copying lm_eval/tasks/wmt2016/metrics.py -> build/lib/lm_eval/tasks/wmt2016
2024-01-31T16:25:25,492   creating build/lib/lm_eval/tasks/translation
2024-01-31T16:25:25,493   copying lm_eval/tasks/translation/utils.py -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:25,496   creating build/lib/lm_eval/tasks/winogrande
2024-01-31T16:25:25,497   copying lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/lib/lm_eval/tasks/winogrande
2024-01-31T16:25:25,499   creating build/lib/lm_eval/tasks/bigbench
2024-01-31T16:25:25,500   copying lm_eval/tasks/bigbench/generate_tasks.py -> build/lib/lm_eval/tasks/bigbench
2024-01-31T16:25:25,503   copying lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/lib/lm_eval/tasks/bigbench
2024-01-31T16:25:25,506   creating build/lib/lm_eval/tasks/webqs
2024-01-31T16:25:25,506   copying lm_eval/tasks/webqs/utils.py -> build/lib/lm_eval/tasks/webqs
2024-01-31T16:25:25,510   creating build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:25,511   copying lm_eval/tasks/blimp/generate_configs.py -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:25,513   creating build/lib/lm_eval/tasks/mathqa
2024-01-31T16:25:25,514   copying lm_eval/tasks/mathqa/utils.py -> build/lib/lm_eval/tasks/mathqa
2024-01-31T16:25:25,516   creating build/lib/lm_eval/tasks/hellaswag
2024-01-31T16:25:25,517   copying lm_eval/tasks/hellaswag/utils.py -> build/lib/lm_eval/tasks/hellaswag
2024-01-31T16:25:25,520   creating build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:25,521   copying lm_eval/tasks/csatqa/_generate_configs.py -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:25,522   copying lm_eval/tasks/csatqa/utils.py -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:25,525   creating build/lib/lm_eval/tasks/wikitext
2024-01-31T16:25:25,526   copying lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/lib/lm_eval/tasks/wikitext
2024-01-31T16:25:25,529   creating build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:25,529   copying lm_eval/tasks/belebele/_generate_configs.py -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:25,532   creating build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:25,532   copying lm_eval/tasks/kobest/utils.py -> build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:25,535   creating build/lib/lm_eval/tasks/squadv2
2024-01-31T16:25:25,536   copying lm_eval/tasks/squadv2/task.py -> build/lib/lm_eval/tasks/squadv2
2024-01-31T16:25:25,539   creating build/lib/lm_eval/tasks/coqa
2024-01-31T16:25:25,540   copying lm_eval/tasks/coqa/utils.py -> build/lib/lm_eval/tasks/coqa
2024-01-31T16:25:25,542   creating build/lib/lm_eval/tasks/toxigen
2024-01-31T16:25:25,543   copying lm_eval/tasks/toxigen/utils.py -> build/lib/lm_eval/tasks/toxigen
2024-01-31T16:25:25,546   creating build/lib/lm_eval/tasks/mgsm
2024-01-31T16:25:25,547   copying lm_eval/tasks/mgsm/utils.py -> build/lib/lm_eval/tasks/mgsm
2024-01-31T16:25:25,551   creating build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:25,552   copying lm_eval/tasks/minerva_math/utils.py -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:25,554   creating build/lib/lm_eval/tasks/scrolls
2024-01-31T16:25:25,555   copying lm_eval/tasks/scrolls/task.py -> build/lib/lm_eval/tasks/scrolls
2024-01-31T16:25:25,559   creating build/lib/lm_eval/tasks/logiqa2
2024-01-31T16:25:25,560   copying lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/lib/lm_eval/tasks/logiqa2
2024-01-31T16:25:25,562   creating build/lib/lm_eval/tasks/medqa
2024-01-31T16:25:25,563   copying lm_eval/tasks/medqa/preprocess_medqa.py -> build/lib/lm_eval/tasks/medqa
2024-01-31T16:25:25,567   creating build/lib/lm_eval/tasks/drop
2024-01-31T16:25:25,567   copying lm_eval/tasks/drop/utils.py -> build/lib/lm_eval/tasks/drop
2024-01-31T16:25:25,571   creating build/lib/lm_eval/tasks/qasper
2024-01-31T16:25:25,572   copying lm_eval/tasks/qasper/metrics.py -> build/lib/lm_eval/tasks/qasper
2024-01-31T16:25:25,574   copying lm_eval/tasks/qasper/utils.py -> build/lib/lm_eval/tasks/qasper
2024-01-31T16:25:25,576   creating build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:25,577   copying lm_eval/tasks/cmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:25,580   creating build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:25,581   copying lm_eval/tasks/paws-x/_generate_config.py -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:25,583   creating build/lib/lm_eval/tasks/mmlu
2024-01-31T16:25:25,584   copying lm_eval/tasks/mmlu/_generate_configs.py -> build/lib/lm_eval/tasks/mmlu
2024-01-31T16:25:25,587   creating build/lib/lm_eval/tasks/bbh
2024-01-31T16:25:25,588   copying lm_eval/tasks/bbh/_generate_configs.py -> build/lib/lm_eval/tasks/bbh
2024-01-31T16:25:25,592   creating build/lib/lm_eval/tasks/qa4mre
2024-01-31T16:25:25,593   copying lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/lib/lm_eval/tasks/qa4mre
2024-01-31T16:25:25,596   creating build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:25,596   copying lm_eval/tasks/xwinograd/utils.py -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:25,599   creating build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:25,600   copying lm_eval/tasks/xnli/utils.py -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:25,603   creating build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:25,604   copying lm_eval/tasks/xcopa/utils.py -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:25,606   creating build/lib/lm_eval/tasks/wsc273
2024-01-31T16:25:25,607   copying lm_eval/tasks/wsc273/utils.py -> build/lib/lm_eval/tasks/wsc273
2024-01-31T16:25:25,609   creating build/lib/lm_eval/tasks/race
2024-01-31T16:25:25,610   copying lm_eval/tasks/race/preprocess_race.py -> build/lib/lm_eval/tasks/race
2024-01-31T16:25:25,612   creating build/lib/lm_eval/tasks/logiqa
2024-01-31T16:25:25,613   copying lm_eval/tasks/logiqa/utils_logiqa.py -> build/lib/lm_eval/tasks/logiqa
2024-01-31T16:25:25,615   creating build/lib/lm_eval/tasks/realtoxicityprompts
2024-01-31T16:25:25,616   copying lm_eval/tasks/realtoxicityprompts/metric.py -> build/lib/lm_eval/tasks/realtoxicityprompts
2024-01-31T16:25:25,619   creating build/lib/lm_eval/tasks/model_written_evals
2024-01-31T16:25:25,619   creating build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:25,620   copying lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:25,623   creating build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:25,624   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:25,630   creating build/lib/lm_eval/tasks/code_x_glue
2024-01-31T16:25:25,630   creating build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:25,631   copying lm_eval/tasks/code_x_glue/code-text/utils.py -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:25,633   copying lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:25,637   creating build/lib/lm_eval/tasks/okapi
2024-01-31T16:25:25,637   creating build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:25,639   copying lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:25,642   creating build/lib/lm_eval/tasks/glue
2024-01-31T16:25:25,643   creating build/lib/lm_eval/tasks/glue/mnli
2024-01-31T16:25:25,644   copying lm_eval/tasks/glue/mnli/utils.py -> build/lib/lm_eval/tasks/glue/mnli
2024-01-31T16:25:25,653   creating build/lib/lm_eval/tasks/super_glue
2024-01-31T16:25:25,653   creating build/lib/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:25,654   copying lm_eval/tasks/super_glue/copa/utils.py -> build/lib/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:25,657   creating build/lib/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:25,657   copying lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:25,660   creating build/lib/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:25,661   copying lm_eval/tasks/super_glue/cb/aggregate.py -> build/lib/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:25,663   copying lm_eval/tasks/super_glue/cb/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:25,665   creating build/lib/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:25,666   copying lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/lib/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:25,668   copying lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:25,670   creating build/lib/lm_eval/tasks/super_glue/record
2024-01-31T16:25:25,671   copying lm_eval/tasks/super_glue/record/util.py -> build/lib/lm_eval/tasks/super_glue/record
2024-01-31T16:25:25,673   copying lm_eval/tasks/super_glue/record/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/record
2024-01-31T16:25:25,676   running egg_info
2024-01-31T16:25:25,680   writing lm_eval.egg-info/PKG-INFO
2024-01-31T16:25:25,697   writing dependency_links to lm_eval.egg-info/dependency_links.txt
2024-01-31T16:25:25,699   writing entry points to lm_eval.egg-info/entry_points.txt
2024-01-31T16:25:25,708   writing requirements to lm_eval.egg-info/requires.txt
2024-01-31T16:25:25,709   writing top-level names to lm_eval.egg-info/top_level.txt
2024-01-31T16:25:26,080   reading manifest file 'lm_eval.egg-info/SOURCES.txt'
2024-01-31T16:25:26,118   adding license file 'LICENSE.md'
2024-01-31T16:25:26,198   writing manifest file 'lm_eval.egg-info/SOURCES.txt'
2024-01-31T16:25:26,571   copying lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:26,573   copying lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:26,575   copying lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:26,577   copying lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:26,579   copying lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:26,581   copying lm_eval/tasks/medmcqa/medmcqa.yaml -> build/lib/lm_eval/tasks/medmcqa
2024-01-31T16:25:26,583   creating build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,584   copying lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,586   copying lm_eval/tasks/pile/pile_gutenberg.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,587   copying lm_eval/tasks/pile/pile_pile-cc.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,589   copying lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,591   copying lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,593   copying lm_eval/tasks/pile/pile_github.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,595   copying lm_eval/tasks/pile/pile_hackernews.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,597   copying lm_eval/tasks/pile/pile_books3.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,599   copying lm_eval/tasks/pile/pile_uspto.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,601   copying lm_eval/tasks/pile/pile_arxiv.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,603   copying lm_eval/tasks/pile/pile_wikipedia.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,604   copying lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,606   copying lm_eval/tasks/pile/pile_freelaw.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,608   copying lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,610   copying lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,612   copying lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,614   copying lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,616   copying lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,617   copying lm_eval/tasks/pile/pile_europarl.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,619   copying lm_eval/tasks/pile/pile_enron.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,621   copying lm_eval/tasks/pile/pile_stackexchange.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,623   copying lm_eval/tasks/pile/pile_philpapers.yaml -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:26,625   copying lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/lib/lm_eval/tasks/truthfulqa
2024-01-31T16:25:26,628   copying lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/lib/lm_eval/tasks/truthfulqa
2024-01-31T16:25:26,629   copying lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/lib/lm_eval/tasks/truthfulqa
2024-01-31T16:25:26,631   copying lm_eval/tasks/mutual/multual_plus.yaml -> build/lib/lm_eval/tasks/mutual
2024-01-31T16:25:26,633   copying lm_eval/tasks/mutual/mutual.yaml -> build/lib/lm_eval/tasks/mutual
2024-01-31T16:25:26,635   creating build/lib/lm_eval/tasks/swag
2024-01-31T16:25:26,636   copying lm_eval/tasks/swag/swag.yaml -> build/lib/lm_eval/tasks/swag
2024-01-31T16:25:26,637   copying lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,639   copying lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,641   copying lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,643   copying lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,645   copying lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,646   copying lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,648   copying lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,650   copying lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,652   copying lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,654   copying lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,656   copying lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,657   copying lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,659   copying lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,661   copying lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,663   copying lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,665   copying lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,667   copying lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,669   copying lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,671   copying lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,672   copying lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,674   copying lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,676   copying lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,678   copying lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,680   copying lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,682   copying lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,684   copying lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,686   copying lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,688   copying lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,690   copying lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,691   copying lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,693   copying lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,695   copying lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,697   copying lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,698   copying lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,700   copying lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,702   copying lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,704   copying lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,706   copying lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,708   copying lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,709   copying lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,711   copying lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,713   copying lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,715   copying lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,717   copying lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,719   copying lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,721   copying lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,723   copying lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,725   copying lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,727   copying lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,729   copying lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,731   copying lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,733   copying lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:26,735   copying lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,737   copying lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,739   copying lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,741   copying lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,743   copying lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,745   copying lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,747   copying lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,749   copying lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,751   copying lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,753   copying lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,755   copying lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,757   copying lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,759   copying lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,761   copying lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,763   copying lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,764   copying lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,766   copying lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,768   copying lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,770   copying lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,772   copying lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,774   copying lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,776   copying lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:26,777   copying lm_eval/tasks/ifeval/ifeval.yaml -> build/lib/lm_eval/tasks/ifeval
2024-01-31T16:25:26,779   creating build/lib/lm_eval/tasks/storycloze
2024-01-31T16:25:26,780   copying lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/lib/lm_eval/tasks/storycloze
2024-01-31T16:25:26,782   copying lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/lib/lm_eval/tasks/storycloze
2024-01-31T16:25:26,784   copying lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/lib/lm_eval/tasks/pubmedqa
2024-01-31T16:25:26,786   copying lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/lib/lm_eval/tasks/wmt2016
2024-01-31T16:25:26,788   copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,790   copying lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,792   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,794   copying lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,796   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,797   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,799   copying lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,801   copying lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,803   copying lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,805   copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,807   copying lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,809   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,811   copying lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,813   copying lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,814   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,816   copying lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,818   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,820   copying lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,822   copying lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,824   copying lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,826   copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,827   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,829   copying lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,831   copying lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,833   copying lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,835   copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,837   copying lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,839   copying lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,841   copying lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,843   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,845   copying lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,847   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,849   copying lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,851   copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,853   copying lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,855   copying lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,857   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,859   copying lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,861   copying lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,862   copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,864   copying lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,866   copying lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,868   copying lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,870   copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,872   copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,873   copying lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,875   copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,877   copying lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,880   copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,881   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,883   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,885   copying lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,887   copying lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,889   copying lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,891   copying lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,893   copying lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,895   copying lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,897   copying lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,899   copying lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,901   copying lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,903   copying lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,905   copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,907   copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,909   copying lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,911   copying lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,913   copying lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,914   copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,916   copying lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,918   copying lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,920   copying lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,922   copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,924   copying lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,926   copying lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,928   copying lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,930   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,932   copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,934   copying lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,936   copying lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,938   copying lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,940   copying lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,942   copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,944   copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,946   copying lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,948   copying lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,950   copying lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,952   copying lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,953   copying lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,955   copying lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,957   copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,959   copying lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,961   copying lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,963   copying lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,964   copying lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,966   copying lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,968   copying lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,970   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,972   copying lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,974   copying lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,976   copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,978   copying lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,980   copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,981   copying lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,983   copying lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,985   copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,987   copying lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,989   copying lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,991   copying lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,993   copying lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,995   copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,997   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:26,998   copying lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,000   copying lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,002   copying lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,004   copying lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,006   copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,008   copying lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,010   copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,012   copying lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,014   copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,016   copying lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,018   copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,020   copying lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,022   copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,023   copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,025   copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,027   copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,029   copying lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,031   copying lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,033   copying lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,035   copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,037   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,039   copying lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,041   copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,043   copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,045   copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:27,047   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,049   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,050   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,052   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,054   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,056   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,058   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,060   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,062   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,064   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,066   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,068   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,070   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,071   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,073   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,075   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,077   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,079   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,081   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,083   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,085   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,087   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,089   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,091   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,092   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,094   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,096   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,098   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,100   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,102   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,104   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,106   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,108   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,110   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,112   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,114   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,116   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,118   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,120   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,122   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,124   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,126   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,128   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,130   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,132   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,133   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,135   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,137   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,139   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:27,141   creating build/lib/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:27,142   copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:27,144   copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:27,146   copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:27,147   copying lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,149   copying lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,151   copying lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,153   copying lm_eval/tasks/translation/wmt16_en-de.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,155   copying lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,157   copying lm_eval/tasks/translation/wmt16_de-en.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,159   copying lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,162   copying lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:27,164   copying lm_eval/tasks/winogrande/default.yaml -> build/lib/lm_eval/tasks/winogrande
2024-01-31T16:25:27,166   creating build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,167   copying lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,170   copying lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,171   copying lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,173   copying lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,175   copying lm_eval/tasks/bigbench/multiple_choice/gem.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,177   copying lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,179   copying lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,181   copying lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,183   copying lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,185   copying lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,187   copying lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,189   copying lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,190   copying lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,192   copying lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,194   copying lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,196   copying lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_transliterate.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,198   copying lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,200   copying lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,202   copying lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,204   copying lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,206   copying lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,209   copying lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,211   copying lm_eval/tasks/bigbench/multiple_choice/language_games.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,213   copying lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,215   copying lm_eval/tasks/bigbench/multiple_choice/linguistics_puzzles.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,217   copying lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,219   copying lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,221   copying lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,223   copying lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,225   copying lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,227   copying lm_eval/tasks/bigbench/multiple_choice/minute_mysteries_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,229   copying lm_eval/tasks/bigbench/multiple_choice/natural_instructions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,231   copying lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,233   copying lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,234   copying lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,236   copying lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,238   copying lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,240   copying lm_eval/tasks/bigbench/multiple_choice/causal_judgement.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,242   copying lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,244   copying lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,246   copying lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,248   copying lm_eval/tasks/bigbench/multiple_choice/conlang_translation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,250   copying lm_eval/tasks/bigbench/multiple_choice/auto_categorization.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,253   copying lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,255   copying lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,257   copying lm_eval/tasks/bigbench/multiple_choice/linguistic_mappings.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,259   copying lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,261   copying lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,263   copying lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,265   copying lm_eval/tasks/bigbench/multiple_choice/list_functions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,267   copying lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,269   copying lm_eval/tasks/bigbench/multiple_choice/gender_inclusive_sentences_german.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,271   copying lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,273   copying lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,275   copying lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,277   copying lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,279   copying lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,281   copying lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,283   copying lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,285   copying lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,286   copying lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,288   copying lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,290   copying lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,292   copying lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,294   copying lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,295   copying lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,297   copying lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,299   copying lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,301   copying lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,303   copying lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,305   copying lm_eval/tasks/bigbench/multiple_choice/matrixshapes.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,307   copying lm_eval/tasks/bigbench/multiple_choice/mult_data_wrangling.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,309   copying lm_eval/tasks/bigbench/multiple_choice/chess_state_tracking.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,310   copying lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,312   copying lm_eval/tasks/bigbench/multiple_choice/word_unscrambling.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,314   copying lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,316   copying lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,318   copying lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,320   copying lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,322   copying lm_eval/tasks/bigbench/multiple_choice/cryptonite.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,324   copying lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,326   copying lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,328   copying lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,330   copying lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,332   copying lm_eval/tasks/bigbench/multiple_choice/simple_text_editing.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,334   copying lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,336   copying lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,338   copying lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,340   copying lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,341   copying lm_eval/tasks/bigbench/multiple_choice/scientific_press_release.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,343   copying lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,345   copying lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,347   copying lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,349   copying lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,351   copying lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,353   copying lm_eval/tasks/bigbench/multiple_choice/disfl_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,354   copying lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,357   copying lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,359   copying lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,361   copying lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,362   copying lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,364   copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,367   copying lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,369   copying lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,370   copying lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,372   copying lm_eval/tasks/bigbench/multiple_choice/sufficient_information.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,374   copying lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,376   copying lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,378   copying lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,380   copying lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_in_context_sparc.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,382   copying lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,384   copying lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,386   copying lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,387   copying lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,389   copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,391   copying lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,393   copying lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,395   copying lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,397   copying lm_eval/tasks/bigbench/multiple_choice/modified_arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,398   copying lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,400   copying lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,402   copying lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,404   copying lm_eval/tasks/bigbench/multiple_choice/bridging_anaphora_resolution_barqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,406   copying lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,408   copying lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,409   copying lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,411   copying lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,413   copying lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,415   copying lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,417   copying lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,419   copying lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,421   copying lm_eval/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,423   copying lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,425   copying lm_eval/tasks/bigbench/multiple_choice/few_shot_nlg.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,427   copying lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,429   copying lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,431   copying lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,432   copying lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,434   copying lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,436   copying lm_eval/tasks/bigbench/multiple_choice/ascii_word_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,438   copying lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,440   copying lm_eval/tasks/bigbench/multiple_choice/operators.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,442   copying lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,444   copying lm_eval/tasks/bigbench/multiple_choice/tense.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,446   copying lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,447   copying lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,449   copying lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,451   copying lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,453   copying lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,455   copying lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,457   copying lm_eval/tasks/bigbench/multiple_choice/codenames.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,459   copying lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_spider.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,461   copying lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,463   copying lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,464   copying lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,466   copying lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,468   copying lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,470   copying lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,473   copying lm_eval/tasks/bigbench/multiple_choice/chinese_remainder_theorem.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,474   copying lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,476   copying lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,478   copying lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,480   copying lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,482   copying lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,484   copying lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,486   copying lm_eval/tasks/bigbench/multiple_choice/hindi_question_answering.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,488   copying lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,490   copying lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:27,492   creating build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,493   copying lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,495   copying lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,496   copying lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,498   copying lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,500   copying lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,502   copying lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,504   copying lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,506   copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,507   copying lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,509   copying lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,511   copying lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,513   copying lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,515   copying lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,517   copying lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,519   copying lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,521   copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,523   copying lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,525   copying lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,526   copying lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,528   copying lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,530   copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,532   copying lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,534   copying lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,536   copying lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,538   copying lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,540   copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,542   copying lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,543   copying lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,545   copying lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,547   copying lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,549   copying lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,551   copying lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,553   copying lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,554   copying lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,556   copying lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,558   copying lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,560   copying lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,562   copying lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,563   copying lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,566   copying lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,567   copying lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,570   copying lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,572   copying lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,574   copying lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,575   copying lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,578   copying lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,580   copying lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,581   copying lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,583   copying lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,586   copying lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,588   copying lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,589   copying lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,591   copying lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,593   copying lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,595   copying lm_eval/tasks/bigbench/generate_until/color.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,597   copying lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,600   copying lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,602   copying lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,604   copying lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,605   copying lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,607   copying lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,609   copying lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,611   copying lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,613   copying lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,615   copying lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,617   copying lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,619   copying lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,622   copying lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,624   copying lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,625   copying lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,627   copying lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,630   copying lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,632   copying lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,634   copying lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,636   copying lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,638   copying lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,640   copying lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,642   copying lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,644   copying lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,646   copying lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,648   copying lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,650   copying lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,652   copying lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,654   copying lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,656   copying lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,658   copying lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,660   copying lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,662   copying lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,664   copying lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,666   copying lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,668   copying lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,670   copying lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,672   copying lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,674   copying lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,676   copying lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,678   copying lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,680   copying lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,682   copying lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,684   copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,686   copying lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,688   copying lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,690   copying lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,692   copying lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,694   copying lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,696   copying lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,698   copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,700   copying lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,702   copying lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,704   copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,706   copying lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,708   copying lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,710   copying lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,712   copying lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,714   copying lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,716   copying lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,718   copying lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,720   copying lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,722   copying lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,724   copying lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,726   copying lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,728   copying lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,730   copying lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,732   copying lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,734   copying lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,736   copying lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,738   copying lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,740   copying lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,742   copying lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,744   copying lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,746   copying lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,748   copying lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,750   copying lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,752   copying lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,754   copying lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,756   copying lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,758   copying lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,760   copying lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,762   copying lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,764   copying lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,767   copying lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,769   copying lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,771   copying lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,773   copying lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,775   copying lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,777   copying lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,779   copying lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,781   copying lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,783   copying lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,785   copying lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,787   copying lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,789   copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,791   copying lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,793   copying lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,795   copying lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,798   copying lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,800   copying lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,802   copying lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,804   copying lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,806   copying lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,808   copying lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,810   copying lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,812   copying lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,814   copying lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,816   copying lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,818   copying lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,820   copying lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,822   copying lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:27,824   creating build/lib/lm_eval/tasks/triviaqa
2024-01-31T16:25:27,825   copying lm_eval/tasks/triviaqa/default.yaml -> build/lib/lm_eval/tasks/triviaqa
2024-01-31T16:25:27,827   creating build/lib/lm_eval/tasks/polemo2
2024-01-31T16:25:27,828   copying lm_eval/tasks/polemo2/polemo2_out.yaml -> build/lib/lm_eval/tasks/polemo2
2024-01-31T16:25:27,830   copying lm_eval/tasks/polemo2/polemo2_in.yaml -> build/lib/lm_eval/tasks/polemo2
2024-01-31T16:25:27,832   copying lm_eval/tasks/webqs/webqs.yaml -> build/lib/lm_eval/tasks/webqs
2024-01-31T16:25:27,834   creating build/lib/lm_eval/tasks/benchmarks
2024-01-31T16:25:27,835   copying lm_eval/tasks/benchmarks/pythia.yaml -> build/lib/lm_eval/tasks/benchmarks
2024-01-31T16:25:27,837   copying lm_eval/tasks/benchmarks/minerva_math.yaml -> build/lib/lm_eval/tasks/benchmarks
2024-01-31T16:25:27,839   copying lm_eval/tasks/benchmarks/t0_eval.yaml -> build/lib/lm_eval/tasks/benchmarks
2024-01-31T16:25:27,841   creating build/lib/lm_eval/tasks/benchmarks/multimedqa
2024-01-31T16:25:27,842   copying lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/lib/lm_eval/tasks/benchmarks/multimedqa
2024-01-31T16:25:27,844   creating build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,845   copying lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,847   copying lm_eval/tasks/benchmarks/flan/flan_rte.yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,849   copying lm_eval/tasks/benchmarks/flan/flan_anli.yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,851   copying lm_eval/tasks/benchmarks/flan/flan_boolq.yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,853   copying lm_eval/tasks/benchmarks/flan/flan_arc.yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,855   copying lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,857   copying lm_eval/tasks/benchmarks/flan/flan_cot.yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:27,859   creating build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:27,860   copying lm_eval/tasks/benchmarks/flan/prompt_templates/arc.yaml -> build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:27,862   copying lm_eval/tasks/benchmarks/flan/prompt_templates/anli.yaml -> build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:27,864   copying lm_eval/tasks/benchmarks/flan/prompt_templates/boolq.yaml -> build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:27,866   copying lm_eval/tasks/benchmarks/flan/prompt_templates/rte.yaml -> build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:27,868   creating build/lib/lm_eval/tasks/babi
2024-01-31T16:25:27,869   copying lm_eval/tasks/babi/babi.yaml -> build/lib/lm_eval/tasks/babi
2024-01-31T16:25:27,871   creating build/lib/lm_eval/tasks/arc
2024-01-31T16:25:27,872   copying lm_eval/tasks/arc/arc_challenge.yaml -> build/lib/lm_eval/tasks/arc
2024-01-31T16:25:27,874   copying lm_eval/tasks/arc/arc_easy.yaml -> build/lib/lm_eval/tasks/arc
2024-01-31T16:25:27,877   copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,879   copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,881   copying lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,883   copying lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,885   copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,887   copying lm_eval/tasks/blimp/npi_present_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,889   copying lm_eval/tasks/blimp/npi_present_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,891   copying lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,893   copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,895   copying lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,897   copying lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,898   copying lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,900   copying lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,902   copying lm_eval/tasks/blimp/causative.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,904   copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,906   copying lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,908   copying lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,910   copying lm_eval/tasks/blimp/wh_island.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,912   copying lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,914   copying lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,916   copying lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,918   copying lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,920   copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,922   copying lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,924   copying lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,926   copying lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,928   copying lm_eval/tasks/blimp/inchoative.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,930   copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,932   copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,934   copying lm_eval/tasks/blimp/drop_argument.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,936   copying lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,938   copying lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,940   copying lm_eval/tasks/blimp/passive_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,942   copying lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,944   copying lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,946   copying lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,948   copying lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,950   copying lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,952   copying lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,954   copying lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,956   copying lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,958   copying lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,960   copying lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,962   copying lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,964   copying lm_eval/tasks/blimp/complex_NP_island.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,966   copying lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,968   copying lm_eval/tasks/blimp/passive_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,970   copying lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,972   copying lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,974   copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,976   copying lm_eval/tasks/blimp/transitive.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,978   copying lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,979   copying lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,981   copying lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,983   copying lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,986   copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,987   copying lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,989   copying lm_eval/tasks/blimp/adjunct_island.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,991   copying lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,993   copying lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,995   copying lm_eval/tasks/blimp/only_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,997   copying lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:27,999   copying lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:28,002   copying lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:28,003   copying lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:28,006   copying lm_eval/tasks/blimp/intransitive.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:28,008   copying lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:28,010   copying lm_eval/tasks/mathqa/mathqa.yaml -> build/lib/lm_eval/tasks/mathqa
2024-01-31T16:25:28,012   copying lm_eval/tasks/hellaswag/hellaswag.yaml -> build/lib/lm_eval/tasks/hellaswag
2024-01-31T16:25:28,014   copying lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:28,016   copying lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:28,018   copying lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:28,020   copying lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:28,022   copying lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:28,024   copying lm_eval/tasks/csatqa/csatqa_li.yaml -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:28,026   creating build/lib/lm_eval/tasks/lambada
2024-01-31T16:25:28,027   copying lm_eval/tasks/lambada/lambada_openai.yaml -> build/lib/lm_eval/tasks/lambada
2024-01-31T16:25:28,029   copying lm_eval/tasks/lambada/lambada_standard.yaml -> build/lib/lm_eval/tasks/lambada
2024-01-31T16:25:28,031   copying lm_eval/tasks/wikitext/wikitext.yaml -> build/lib/lm_eval/tasks/wikitext
2024-01-31T16:25:28,033   creating build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,034   copying lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,036   copying lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,038   copying lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,039   copying lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,041   copying lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,043   copying lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,045   copying lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,047   copying lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,049   copying lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,051   copying lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:28,053   copying lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,055   copying lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,057   copying lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,059   copying lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,061   copying lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,063   copying lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,065   copying lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,067   copying lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,070   copying lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,072   copying lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,074   copying lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,076   copying lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,078   copying lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,080   copying lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,082   copying lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,084   copying lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,086   copying lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,088   copying lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,090   copying lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,092   copying lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,094   copying lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,096   copying lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,098   copying lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,100   copying lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,102   copying lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,104   copying lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,106   copying lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,108   copying lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,110   copying lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,112   copying lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,114   copying lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,116   copying lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,118   copying lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,120   copying lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,122   copying lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,123   copying lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,125   copying lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,127   copying lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,129   copying lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,131   copying lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,138   copying lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,140   copying lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,142   copying lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,144   copying lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,146   copying lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,148   copying lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,150   copying lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,152   copying lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,154   copying lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,156   copying lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,158   copying lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,161   copying lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,162   copying lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,165   copying lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,166   copying lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,168   copying lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,170   copying lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,173   copying lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,175   copying lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,177   copying lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,179   copying lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,181   copying lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,183   copying lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,185   copying lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,187   copying lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,189   copying lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,191   copying lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,193   copying lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,195   copying lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,197   copying lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,199   copying lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,202   copying lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,204   copying lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,206   copying lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,208   copying lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,210   copying lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,212   copying lm_eval/tasks/belebele/belebele_default.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,214   copying lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,216   copying lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,218   copying lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,220   copying lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,222   copying lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,224   copying lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,226   copying lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,228   copying lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,230   copying lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,232   copying lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,234   copying lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,236   copying lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,238   copying lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,240   copying lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,242   copying lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,244   copying lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,246   copying lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,248   copying lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,250   copying lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,252   copying lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,254   copying lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,256   copying lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,258   copying lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,260   copying lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,262   copying lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,264   copying lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,266   copying lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,268   copying lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,270   copying lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,272   copying lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,274   copying lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,276   copying lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,278   copying lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,280   copying lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,282   copying lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,284   copying lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,286   copying lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,288   copying lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,290   copying lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,292   copying lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,294   copying lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,296   copying lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,298   copying lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,300   copying lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,302   copying lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,304   copying lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:28,306   copying lm_eval/tasks/kobest/kobest_boolq.yaml -> build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:28,308   copying lm_eval/tasks/kobest/kobest_copa.yaml -> build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:28,310   copying lm_eval/tasks/kobest/kobest_wic.yaml -> build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:28,312   copying lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:28,314   copying lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:28,316   copying lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:28,318   copying lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:28,320   copying lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:28,322   copying lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:28,324   copying lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:28,326   copying lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:28,328   copying lm_eval/tasks/coqa/default.yaml -> build/lib/lm_eval/tasks/coqa
2024-01-31T16:25:28,330   copying lm_eval/tasks/toxigen/toxigen.yaml -> build/lib/lm_eval/tasks/toxigen
2024-01-31T16:25:28,332   creating build/lib/lm_eval/tasks/nq_open
2024-01-31T16:25:28,333   copying lm_eval/tasks/nq_open/nq_open.yaml -> build/lib/lm_eval/tasks/nq_open
2024-01-31T16:25:28,335   creating build/lib/lm_eval/tasks/gsm8k
2024-01-31T16:25:28,336   copying lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/lib/lm_eval/tasks/gsm8k
2024-01-31T16:25:28,338   copying lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/lib/lm_eval/tasks/gsm8k
2024-01-31T16:25:28,340   copying lm_eval/tasks/gsm8k/gsm8k.yaml -> build/lib/lm_eval/tasks/gsm8k
2024-01-31T16:25:28,342   creating build/lib/lm_eval/tasks/asdiv
2024-01-31T16:25:28,343   copying lm_eval/tasks/asdiv/default.yaml -> build/lib/lm_eval/tasks/asdiv
2024-01-31T16:25:28,345   creating build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,346   copying lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,348   copying lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,350   copying lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,352   copying lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,354   copying lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,356   copying lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,359   copying lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,361   copying lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,363   copying lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,366   copying lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,369   copying lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:28,371   creating build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,372   copying lm_eval/tasks/mgsm/en_cot/mgsm_te_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,374   copying lm_eval/tasks/mgsm/en_cot/mgsm_sw_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,376   copying lm_eval/tasks/mgsm/en_cot/mgsm_ru_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,378   copying lm_eval/tasks/mgsm/en_cot/mgsm_de_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,380   copying lm_eval/tasks/mgsm/en_cot/mgsm_es_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,382   copying lm_eval/tasks/mgsm/en_cot/mgsm_en_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,384   copying lm_eval/tasks/mgsm/en_cot/mgsm_zh_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,386   copying lm_eval/tasks/mgsm/en_cot/mgsm_fr_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,388   copying lm_eval/tasks/mgsm/en_cot/mgsm_bn_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,390   copying lm_eval/tasks/mgsm/en_cot/mgsm_ja_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,392   copying lm_eval/tasks/mgsm/en_cot/mgsm_th_en-cot.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:28,394   creating build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,395   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_de.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,397   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_sw.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,399   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_en.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,402   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_fr.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,404   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_es.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,406   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_zh.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,409   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_th.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,411   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_ja.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,413   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_ru.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,415   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_te.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,417   copying lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_bn.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:28,419   creating build/lib/lm_eval/tasks/sciq
2024-01-31T16:25:28,420   copying lm_eval/tasks/sciq/sciq.yaml -> build/lib/lm_eval/tasks/sciq
2024-01-31T16:25:28,423   creating build/lib/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:28,424   copying lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/lib/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:28,426   copying lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/lib/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:28,428   copying lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/lib/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:28,430   copying lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/lib/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:28,433   copying lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/lib/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:28,435   creating build/lib/lm_eval/tasks/piqa
2024-01-31T16:25:28,436   copying lm_eval/tasks/piqa/piqa.yaml -> build/lib/lm_eval/tasks/piqa
2024-01-31T16:25:28,438   copying lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:28,440   copying lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:28,443   copying lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:28,445   copying lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:28,447   copying lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:28,450   copying lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:28,453   copying lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:28,456   copying lm_eval/tasks/scrolls/scrolls.yaml -> build/lib/lm_eval/tasks/scrolls
2024-01-31T16:25:28,458   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,460   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,462   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,464   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,466   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,468   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,470   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,471   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,473   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,475   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,477   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,480   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,482   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,484   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,486   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,488   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,490   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,492   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,494   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,496   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,498   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,500   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,502   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,504   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,506   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,508   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,510   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,513   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,515   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,517   copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:28,519   copying lm_eval/tasks/logiqa2/logieval.yaml -> build/lib/lm_eval/tasks/logiqa2
2024-01-31T16:25:28,521   copying lm_eval/tasks/logiqa2/logiqa2.yaml -> build/lib/lm_eval/tasks/logiqa2
2024-01-31T16:25:28,523   creating build/lib/lm_eval/tasks/siqa
2024-01-31T16:25:28,524   copying lm_eval/tasks/siqa/siqa.yaml -> build/lib/lm_eval/tasks/siqa
2024-01-31T16:25:28,526   creating build/lib/lm_eval/tasks/fld
2024-01-31T16:25:28,527   copying lm_eval/tasks/fld/fld_star.yaml -> build/lib/lm_eval/tasks/fld
2024-01-31T16:25:28,529   copying lm_eval/tasks/fld/fld_default.yaml -> build/lib/lm_eval/tasks/fld
2024-01-31T16:25:28,531   copying lm_eval/tasks/medqa/medqa.yaml -> build/lib/lm_eval/tasks/medqa
2024-01-31T16:25:28,533   creating build/lib/lm_eval/tasks/glue/sst2
2024-01-31T16:25:28,534   copying lm_eval/tasks/glue/sst2/default.yaml -> build/lib/lm_eval/tasks/glue/sst2
2024-01-31T16:25:28,535   creating build/lib/lm_eval/tasks/glue/mrpc
2024-01-31T16:25:28,537   copying lm_eval/tasks/glue/mrpc/default.yaml -> build/lib/lm_eval/tasks/glue/mrpc
2024-01-31T16:25:28,538   creating build/lib/lm_eval/tasks/glue/rte
2024-01-31T16:25:28,539   copying lm_eval/tasks/glue/rte/default.yaml -> build/lib/lm_eval/tasks/glue/rte
2024-01-31T16:25:28,541   copying lm_eval/tasks/glue/mnli/default.yaml -> build/lib/lm_eval/tasks/glue/mnli
2024-01-31T16:25:28,543   copying lm_eval/tasks/glue/mnli/mismatch.yaml -> build/lib/lm_eval/tasks/glue/mnli
2024-01-31T16:25:28,545   creating build/lib/lm_eval/tasks/glue/wnli
2024-01-31T16:25:28,546   copying lm_eval/tasks/glue/wnli/default.yaml -> build/lib/lm_eval/tasks/glue/wnli
2024-01-31T16:25:28,549   creating build/lib/lm_eval/tasks/glue/qnli
2024-01-31T16:25:28,550   copying lm_eval/tasks/glue/qnli/default.yaml -> build/lib/lm_eval/tasks/glue/qnli
2024-01-31T16:25:28,552   creating build/lib/lm_eval/tasks/glue/qqp
2024-01-31T16:25:28,553   copying lm_eval/tasks/glue/qqp/default.yaml -> build/lib/lm_eval/tasks/glue/qqp
2024-01-31T16:25:28,555   creating build/lib/lm_eval/tasks/glue/cola
2024-01-31T16:25:28,556   copying lm_eval/tasks/glue/cola/default.yaml -> build/lib/lm_eval/tasks/glue/cola
2024-01-31T16:25:28,558   creating build/lib/lm_eval/tasks/anli
2024-01-31T16:25:28,559   copying lm_eval/tasks/anli/anli_r1.yaml -> build/lib/lm_eval/tasks/anli
2024-01-31T16:25:28,561   copying lm_eval/tasks/anli/anli_r2.yaml -> build/lib/lm_eval/tasks/anli
2024-01-31T16:25:28,563   copying lm_eval/tasks/anli/anli_r3.yaml -> build/lib/lm_eval/tasks/anli
2024-01-31T16:25:28,565   copying lm_eval/tasks/drop/default.yaml -> build/lib/lm_eval/tasks/drop
2024-01-31T16:25:28,567   creating build/lib/lm_eval/tasks/headqa
2024-01-31T16:25:28,568   copying lm_eval/tasks/headqa/headqa_es.yaml -> build/lib/lm_eval/tasks/headqa
2024-01-31T16:25:28,570   copying lm_eval/tasks/headqa/headqa_en.yaml -> build/lib/lm_eval/tasks/headqa
2024-01-31T16:25:28,572   copying lm_eval/tasks/qasper/bool.yaml -> build/lib/lm_eval/tasks/qasper
2024-01-31T16:25:28,574   copying lm_eval/tasks/qasper/freeform.yaml -> build/lib/lm_eval/tasks/qasper
2024-01-31T16:25:28,577   copying lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,579   copying lm_eval/tasks/cmmlu/cmmlu_default_computer_science.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,581   copying lm_eval/tasks/cmmlu/cmmlu_default_college_medical_statistics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,583   copying lm_eval/tasks/cmmlu/cmmlu_default_college_engineering_hydrology.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,585   copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_civil_service_exam.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,587   copying lm_eval/tasks/cmmlu/cmmlu_default_electrical_engineering.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,589   copying lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,591   copying lm_eval/tasks/cmmlu/cmmlu_default_college_education.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,593   copying lm_eval/tasks/cmmlu/cmmlu_default_legal_and_moral_basis.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,595   copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_foreign_policy.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,597   copying lm_eval/tasks/cmmlu/cmmlu_default_ancient_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,599   copying lm_eval/tasks/cmmlu/cmmlu_default_professional_accounting.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,602   copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_physics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,603   copying lm_eval/tasks/cmmlu/cmmlu_default_arts.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,606   copying lm_eval/tasks/cmmlu/cmmlu_default_business_ethics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,608   copying lm_eval/tasks/cmmlu/cmmlu_default_economics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,610   copying lm_eval/tasks/cmmlu/cmmlu_default_anatomy.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,612   copying lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,613   copying lm_eval/tasks/cmmlu/cmmlu_default_marxist_theory.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,615   copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_geography.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,618   copying lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,620   copying lm_eval/tasks/cmmlu/cmmlu_default_astronomy.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,622   copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_driving_rule.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,624   copying lm_eval/tasks/cmmlu/cmmlu_default_college_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,626   copying lm_eval/tasks/cmmlu/cmmlu_default_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,628   copying lm_eval/tasks/cmmlu/cmmlu_default_professional_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,630   copying lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,632   copying lm_eval/tasks/cmmlu/cmmlu_default_food_science.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,634   copying lm_eval/tasks/cmmlu/cmmlu_default_global_facts.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,636   copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,638   copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_politics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,640   copying lm_eval/tasks/cmmlu/cmmlu_default_college_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,642   copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,644   copying lm_eval/tasks/cmmlu/cmmlu_default_computer_security.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,646   copying lm_eval/tasks/cmmlu/cmmlu_default_machine_learning.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,649   copying lm_eval/tasks/cmmlu/cmmlu_default_education.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,651   copying lm_eval/tasks/cmmlu/cmmlu_default_ethnology.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,653   copying lm_eval/tasks/cmmlu/cmmlu_default_college_law.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,655   copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_biology.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,657   copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_food_culture.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,659   copying lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,661   copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_history.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,663   copying lm_eval/tasks/cmmlu/cmmlu_default_construction_project_management.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,665   copying lm_eval/tasks/cmmlu/cmmlu_default_jurisprudence.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,667   copying lm_eval/tasks/cmmlu/cmmlu_default_professional_law.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,669   copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_commonsense.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,671   copying lm_eval/tasks/cmmlu/cmmlu_default_international_law.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,673   copying lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,675   copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,677   copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_teacher_qualification.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,679   copying lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,682   copying lm_eval/tasks/cmmlu/cmmlu_default_college_actuarial_science.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,685   copying lm_eval/tasks/cmmlu/cmmlu_default_elementary_information_and_technology.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,689   copying lm_eval/tasks/cmmlu/cmmlu_default_journalism.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,693   copying lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,697   copying lm_eval/tasks/cmmlu/cmmlu_default_logical.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,702   copying lm_eval/tasks/cmmlu/cmmlu_default_human_sexuality.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,707   copying lm_eval/tasks/cmmlu/cmmlu_default_agronomy.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,711   copying lm_eval/tasks/cmmlu/cmmlu_default_genetics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,713   copying lm_eval/tasks/cmmlu/cmmlu_default_chinese_literature.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,716   copying lm_eval/tasks/cmmlu/cmmlu_default_philosophy.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,720   copying lm_eval/tasks/cmmlu/cmmlu_default_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,725   copying lm_eval/tasks/cmmlu/cmmlu_default_management.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,730   copying lm_eval/tasks/cmmlu/cmmlu_default_modern_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,736   copying lm_eval/tasks/cmmlu/cmmlu_default_marketing.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,741   copying lm_eval/tasks/cmmlu/cmmlu_default_conceptual_physics.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,743   copying lm_eval/tasks/cmmlu/cmmlu_default_nutrition.yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:28,745   copying lm_eval/tasks/paws-x/paws_fr.yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:28,747   copying lm_eval/tasks/paws-x/paws_ja.yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:28,749   copying lm_eval/tasks/paws-x/paws_es.yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:28,751   copying lm_eval/tasks/paws-x/paws_en.yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:28,753   copying lm_eval/tasks/paws-x/paws_de.yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:28,755   copying lm_eval/tasks/paws-x/paws_ko.yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:28,757   copying lm_eval/tasks/paws-x/paws_zh.yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:28,759   creating build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,760   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,762   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,764   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,766   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,768   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,770   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,772   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,774   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,776   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,778   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,780   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,782   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,784   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,786   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,788   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,790   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,792   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,794   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,796   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,798   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,800   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,802   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,804   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,806   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,808   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,810   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,812   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,814   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,816   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,818   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,820   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,822   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,824   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,826   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,828   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,830   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,832   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,834   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,836   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,838   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,840   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,842   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,844   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,846   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,848   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,850   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,852   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,854   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,856   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,858   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,860   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,862   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,864   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,866   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,868   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,870   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,872   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,874   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:28,875   creating build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,876   copying lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,878   copying lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,880   copying lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,882   copying lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,884   copying lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,886   copying lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,888   copying lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,890   copying lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,892   copying lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,894   copying lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,896   copying lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,898   copying lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,900   copying lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,902   copying lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,904   copying lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,906   copying lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,908   copying lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,910   copying lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,912   copying lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,914   copying lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,916   copying lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,918   copying lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,920   copying lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,922   copying lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,924   copying lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,926   copying lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,929   copying lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,931   copying lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,933   copying lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,935   copying lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,937   copying lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,939   copying lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,941   copying lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,943   copying lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,945   copying lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,946   copying lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,949   copying lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,951   copying lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,953   copying lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,955   copying lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,957   copying lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,959   copying lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,961   copying lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,963   copying lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,965   copying lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,967   copying lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,969   copying lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,971   copying lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,973   copying lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,975   copying lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,977   copying lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,979   copying lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,981   copying lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,983   copying lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,985   copying lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,987   copying lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,989   copying lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,991   copying lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:28,993   creating build/lib/lm_eval/tasks/mmlu/flan_n_shot
2024-01-31T16:25:28,994   creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:28,995   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:28,997   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:28,999   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,001   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,003   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,005   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,007   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,009   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,011   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,013   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,015   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,017   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,019   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,021   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,023   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,025   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,027   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,030   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,031   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,034   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,036   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,038   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,040   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,042   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,045   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,047   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,049   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,051   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,053   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,055   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,057   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,059   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,061   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,063   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,065   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,067   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,069   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,071   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,073   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,075   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,078   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,080   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,082   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,084   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,086   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,088   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,091   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,094   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,096   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,099   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,102   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,105   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,108   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,111   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,114   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,118   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,121   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,124   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:29,128   creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,130   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,132   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,135   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,137   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,140   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,142   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,144   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,146   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,148   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,150   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,153   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,155   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,157   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,159   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,164   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,166   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,168   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,170   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,172   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,174   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,176   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,178   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,180   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,182   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,184   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,186   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,188   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,190   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,192   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,194   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,196   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,198   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,200   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,202   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,204   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,206   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,208   copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,210   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,212   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,214   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,216   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,218   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,220   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,222   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,224   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,226   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,228   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,230   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,232   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,234   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,235   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,238   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,240   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,242   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,244   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,246   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,248   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,250   copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:29,252   creating build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,253   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,255   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,257   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,260   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,262   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,264   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,267   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,269   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,271   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,273   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,275   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,277   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,280   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,282   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,284   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,286   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,288   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,290   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,292   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,294   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,296   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,298   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,301   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,303   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,305   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,308   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,310   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,313   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,315   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,318   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,320   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,323   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,325   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,327   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,330   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,332   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,335   copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,337   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,339   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,342   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,344   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,347   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,349   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,351   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,353   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,356   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,358   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,360   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,362   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,364   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,366   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,369   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,371   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,373   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,376   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,378   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,380   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,382   copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:29,385   creating build/lib/lm_eval/tasks/prost
2024-01-31T16:25:29,386   copying lm_eval/tasks/prost/corypaik_prost.yaml -> build/lib/lm_eval/tasks/prost
2024-01-31T16:25:29,388   creating build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,389   copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,391   copying lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,393   copying lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,395   copying lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,398   copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,400   copying lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,402   copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,404   copying lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,407   copying lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,409   copying lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,411   copying lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,413   copying lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,415   copying lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,418   copying lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,420   copying lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,422   copying lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,424   copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,426   copying lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,429   copying lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,431   copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,433   copying lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,435   copying lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,438   copying lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,440   copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,442   copying lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,444   copying lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,447   copying lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:29,449   creating build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,450   copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,452   copying lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,454   copying lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,457   copying lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,459   copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,461   copying lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,463   copying lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,465   copying lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,467   copying lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,469   copying lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,471   copying lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,473   copying lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,475   copying lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,477   copying lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,479   copying lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,481   copying lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,483   copying lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,485   copying lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,487   copying lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,489   copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,491   copying lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,493   copying lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,495   copying lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,497   copying lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,499   copying lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,501   copying lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,503   copying lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:29,505   creating build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,506   copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,508   copying lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,510   copying lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,512   copying lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,514   copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,517   copying lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,519   copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,521   copying lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,523   copying lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,525   copying lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,527   copying lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,529   copying lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,531   copying lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,533   copying lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,535   copying lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,537   copying lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,539   copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,541   copying lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,543   copying lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,545   copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,547   copying lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,549   copying lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,551   copying lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,553   copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,554   copying lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,556   copying lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,558   copying lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:29,561   creating build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,562   copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,564   copying lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,566   copying lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,569   copying lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,571   copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,573   copying lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,575   copying lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,577   copying lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,579   copying lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,581   copying lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,583   copying lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,585   copying lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,587   copying lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,590   copying lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,592   copying lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,594   copying lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,596   copying lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,598   copying lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,600   copying lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,602   copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,605   copying lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,607   copying lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,610   copying lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,612   copying lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,614   copying lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,615   copying lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,617   copying lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:29,619   creating build/lib/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:29,620   copying lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:29,622   copying lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:29,624   creating build/lib/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:29,625   copying lm_eval/tasks/super_glue/boolq/default.yaml -> build/lib/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:29,627   copying lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:29,629   copying lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/lib/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:29,631   copying lm_eval/tasks/super_glue/copa/default.yaml -> build/lib/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:29,634   copying lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:29,636   copying lm_eval/tasks/super_glue/multirc/default.yaml -> build/lib/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:29,638   copying lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:29,640   copying lm_eval/tasks/super_glue/cb/default.yaml -> build/lib/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:29,642   copying lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:29,644   creating build/lib/lm_eval/tasks/super_glue/rte
2024-01-31T16:25:29,645   copying lm_eval/tasks/super_glue/rte/default.yaml -> build/lib/lm_eval/tasks/super_glue/rte
2024-01-31T16:25:29,648   copying lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/rte
2024-01-31T16:25:29,650   copying lm_eval/tasks/super_glue/wsc/default.yaml -> build/lib/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:29,653   copying lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:29,656   copying lm_eval/tasks/super_glue/record/default.yaml -> build/lib/lm_eval/tasks/super_glue/record
2024-01-31T16:25:29,658   copying lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/record
2024-01-31T16:25:29,660   creating build/lib/lm_eval/tasks/super_glue/wic
2024-01-31T16:25:29,661   copying lm_eval/tasks/super_glue/wic/default.yaml -> build/lib/lm_eval/tasks/super_glue/wic
2024-01-31T16:25:29,663   copying lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wic
2024-01-31T16:25:29,665   creating build/lib/lm_eval/tasks/openbookqa
2024-01-31T16:25:29,666   copying lm_eval/tasks/openbookqa/openbookqa.yaml -> build/lib/lm_eval/tasks/openbookqa
2024-01-31T16:25:29,669   copying lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/lib/lm_eval/tasks/qa4mre
2024-01-31T16:25:29,671   copying lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/lib/lm_eval/tasks/qa4mre
2024-01-31T16:25:29,673   copying lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/lib/lm_eval/tasks/qa4mre
2024-01-31T16:25:29,674   creating build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,675   copying lm_eval/tasks/xstorycloze/default_eu.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,677   copying lm_eval/tasks/xstorycloze/default_en.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,679   copying lm_eval/tasks/xstorycloze/default_es.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,681   copying lm_eval/tasks/xstorycloze/default_ru.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,683   copying lm_eval/tasks/xstorycloze/default_sw.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,685   copying lm_eval/tasks/xstorycloze/default_my.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,687   copying lm_eval/tasks/xstorycloze/default_ar.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,689   copying lm_eval/tasks/xstorycloze/default_hi.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,691   copying lm_eval/tasks/xstorycloze/default_te.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,693   copying lm_eval/tasks/xstorycloze/default_id.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,695   copying lm_eval/tasks/xstorycloze/default_zh.yaml -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:29,697   creating build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,698   copying lm_eval/tasks/kmmlu/kmmlu_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,700   copying lm_eval/tasks/kmmlu/kmmlu_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,702   copying lm_eval/tasks/kmmlu/kmmlu_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,704   copying lm_eval/tasks/kmmlu/kmmlu_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,706   copying lm_eval/tasks/kmmlu/kmmlu_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,708   copying lm_eval/tasks/kmmlu/kmmlu_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,710   copying lm_eval/tasks/kmmlu/kmmlu_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,712   copying lm_eval/tasks/kmmlu/kmmlu_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,714   copying lm_eval/tasks/kmmlu/kmmlu_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,715   copying lm_eval/tasks/kmmlu/kmmlu_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,717   copying lm_eval/tasks/kmmlu/kmmlu_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,719   copying lm_eval/tasks/kmmlu/kmmlu_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,721   copying lm_eval/tasks/kmmlu/kmmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,723   copying lm_eval/tasks/kmmlu/kmmlu_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,725   copying lm_eval/tasks/kmmlu/kmmlu_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,727   copying lm_eval/tasks/kmmlu/kmmlu_law.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,730   copying lm_eval/tasks/kmmlu/kmmlu_health.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,732   copying lm_eval/tasks/kmmlu/kmmlu_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,734   copying lm_eval/tasks/kmmlu/kmmlu_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,736   copying lm_eval/tasks/kmmlu/kmmlu_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,738   copying lm_eval/tasks/kmmlu/kmmlu_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,740   copying lm_eval/tasks/kmmlu/kmmlu_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,742   copying lm_eval/tasks/kmmlu/kmmlu_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,743   copying lm_eval/tasks/kmmlu/kmmlu_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,745   copying lm_eval/tasks/kmmlu/kmmlu_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,747   copying lm_eval/tasks/kmmlu/kmmlu_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,749   copying lm_eval/tasks/kmmlu/kmmlu_management.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,751   copying lm_eval/tasks/kmmlu/kmmlu_biology.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,753   copying lm_eval/tasks/kmmlu/kmmlu_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,755   copying lm_eval/tasks/kmmlu/kmmlu_construction.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,757   copying lm_eval/tasks/kmmlu/kmmlu_economics.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,758   copying lm_eval/tasks/kmmlu/kmmlu_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,760   copying lm_eval/tasks/kmmlu/kmmlu_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,762   copying lm_eval/tasks/kmmlu/kmmlu_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,764   copying lm_eval/tasks/kmmlu/kmmlu_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,766   copying lm_eval/tasks/kmmlu/kmmlu_patent.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,768   copying lm_eval/tasks/kmmlu/kmmlu_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,770   copying lm_eval/tasks/kmmlu/kmmlu_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,772   copying lm_eval/tasks/kmmlu/kmmlu_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,774   copying lm_eval/tasks/kmmlu/kmmlu_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,776   copying lm_eval/tasks/kmmlu/kmmlu_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,778   copying lm_eval/tasks/kmmlu/kmmlu_education.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,781   copying lm_eval/tasks/kmmlu/kmmlu_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:29,783   copying lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:29,785   copying lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:29,787   copying lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:29,789   copying lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:29,791   copying lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:29,793   copying lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:29,795   copying lm_eval/tasks/xnli/xnli_bg.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,797   copying lm_eval/tasks/xnli/xnli_en.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,799   copying lm_eval/tasks/xnli/xnli_zh.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,801   copying lm_eval/tasks/xnli/xnli_fr.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,803   copying lm_eval/tasks/xnli/xnli_hi.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,805   copying lm_eval/tasks/xnli/xnli_tr.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,807   copying lm_eval/tasks/xnli/xnli_es.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,809   copying lm_eval/tasks/xnli/xnli_ru.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,811   copying lm_eval/tasks/xnli/xnli_ar.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,813   copying lm_eval/tasks/xnli/xnli_el.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,815   copying lm_eval/tasks/xnli/xnli_sw.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,817   copying lm_eval/tasks/xnli/xnli_de.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,819   copying lm_eval/tasks/xnli/xnli_ur.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,821   copying lm_eval/tasks/xnli/xnli_vi.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,823   copying lm_eval/tasks/xnli/xnli_th.yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:29,824   creating build/lib/lm_eval/tasks/unscramble
2024-01-31T16:25:29,825   copying lm_eval/tasks/unscramble/reversed_words.yaml -> build/lib/lm_eval/tasks/unscramble
2024-01-31T16:25:29,827   copying lm_eval/tasks/unscramble/anagrams1.yaml -> build/lib/lm_eval/tasks/unscramble
2024-01-31T16:25:29,829   copying lm_eval/tasks/unscramble/random_insertion.yaml -> build/lib/lm_eval/tasks/unscramble
2024-01-31T16:25:29,831   copying lm_eval/tasks/unscramble/anagrams2.yaml -> build/lib/lm_eval/tasks/unscramble
2024-01-31T16:25:29,833   copying lm_eval/tasks/unscramble/cycle_letters.yaml -> build/lib/lm_eval/tasks/unscramble
2024-01-31T16:25:29,835   creating build/lib/lm_eval/tasks/mc_taco
2024-01-31T16:25:29,836   copying lm_eval/tasks/mc_taco/default.yaml -> build/lib/lm_eval/tasks/mc_taco
2024-01-31T16:25:29,838   copying lm_eval/tasks/xcopa/default_sw.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,840   copying lm_eval/tasks/xcopa/default_th.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,842   copying lm_eval/tasks/xcopa/default_vi.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,844   copying lm_eval/tasks/xcopa/default_ht.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,846   copying lm_eval/tasks/xcopa/default_id.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,848   copying lm_eval/tasks/xcopa/default_zh.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,850   copying lm_eval/tasks/xcopa/default_tr.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,852   copying lm_eval/tasks/xcopa/default_et.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,854   copying lm_eval/tasks/xcopa/default_qu.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,855   copying lm_eval/tasks/xcopa/default_ta.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,857   copying lm_eval/tasks/xcopa/default_it.yaml -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:29,860   copying lm_eval/tasks/wsc273/default.yaml -> build/lib/lm_eval/tasks/wsc273
2024-01-31T16:25:29,862   copying lm_eval/tasks/race/race.yaml -> build/lib/lm_eval/tasks/race
2024-01-31T16:25:29,863   copying lm_eval/tasks/logiqa/logiqa.yaml -> build/lib/lm_eval/tasks/logiqa
2024-01-31T16:25:29,865   copying lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/lib/lm_eval/tasks/realtoxicityprompts
2024-01-31T16:25:29,868   copying lm_eval/tasks/README.md -> build/lib/lm_eval/tasks
2024-01-31T16:25:29,870   copying lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:29,872   copying lm_eval/tasks/hendrycks_ethics/README.md -> build/lib/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:29,875   copying lm_eval/tasks/pile/README.md -> build/lib/lm_eval/tasks/pile
2024-01-31T16:25:29,877   copying lm_eval/tasks/truthfulqa/README.md -> build/lib/lm_eval/tasks/truthfulqa
2024-01-31T16:25:29,880   copying lm_eval/tasks/mutual/README.md -> build/lib/lm_eval/tasks/mutual
2024-01-31T16:25:29,882   copying lm_eval/tasks/swag/README.md -> build/lib/lm_eval/tasks/swag
2024-01-31T16:25:29,885   copying lm_eval/tasks/ceval/README.md -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:29,887   copying lm_eval/tasks/ceval/_default_ceval_yaml -> build/lib/lm_eval/tasks/ceval
2024-01-31T16:25:29,889   copying lm_eval/tasks/crows_pairs/README.md -> build/lib/lm_eval/tasks/crows_pairs
2024-01-31T16:25:29,893   copying lm_eval/tasks/ifeval/README.md -> build/lib/lm_eval/tasks/ifeval
2024-01-31T16:25:29,896   copying lm_eval/tasks/storycloze/README.md -> build/lib/lm_eval/tasks/storycloze
2024-01-31T16:25:29,898   copying lm_eval/tasks/pubmedqa/README.md -> build/lib/lm_eval/tasks/pubmedqa
2024-01-31T16:25:29,901   copying lm_eval/tasks/wmt2016/README.md -> build/lib/lm_eval/tasks/wmt2016
2024-01-31T16:25:29,902   creating build/lib/lm_eval/tasks/model_written_evals/winogenerated
2024-01-31T16:25:29,903   copying lm_eval/tasks/model_written_evals/winogenerated/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/winogenerated
2024-01-31T16:25:29,906   copying lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:29,908   copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:29,911   copying lm_eval/tasks/translation/wmt_common_yaml -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:29,913   copying lm_eval/tasks/translation/README.md -> build/lib/lm_eval/tasks/translation
2024-01-31T16:25:29,915   copying lm_eval/tasks/winogrande/README.md -> build/lib/lm_eval/tasks/winogrande
2024-01-31T16:25:29,917   copying lm_eval/tasks/bigbench/generate_until_template_yaml -> build/lib/lm_eval/tasks/bigbench
2024-01-31T16:25:29,919   copying lm_eval/tasks/bigbench/README.md -> build/lib/lm_eval/tasks/bigbench
2024-01-31T16:25:29,922   copying lm_eval/tasks/bigbench/multiple_choice_template_yaml -> build/lib/lm_eval/tasks/bigbench
2024-01-31T16:25:29,924   copying lm_eval/tasks/triviaqa/README.md -> build/lib/lm_eval/tasks/triviaqa
2024-01-31T16:25:29,926   copying lm_eval/tasks/polemo2/README.md -> build/lib/lm_eval/tasks/polemo2
2024-01-31T16:25:29,929   copying lm_eval/tasks/webqs/README.md -> build/lib/lm_eval/tasks/webqs
2024-01-31T16:25:29,931   copying lm_eval/tasks/benchmarks/multimedqa/README.md -> build/lib/lm_eval/tasks/benchmarks/multimedqa
2024-01-31T16:25:29,933   copying lm_eval/tasks/benchmarks/flan/flan_held_in_yaml -> build/lib/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:29,935   creating build/lib/lm_eval/tasks/benchmarks/flan/yaml_templates
2024-01-31T16:25:29,936   copying lm_eval/tasks/benchmarks/flan/yaml_templates/cot_template_yaml -> build/lib/lm_eval/tasks/benchmarks/flan/yaml_templates
2024-01-31T16:25:29,938   copying lm_eval/tasks/benchmarks/flan/yaml_templates/held_in_template_yaml -> build/lib/lm_eval/tasks/benchmarks/flan/yaml_templates
2024-01-31T16:25:29,941   copying lm_eval/tasks/babi/README.md -> build/lib/lm_eval/tasks/babi
2024-01-31T16:25:29,943   copying lm_eval/tasks/arc/README.md -> build/lib/lm_eval/tasks/arc
2024-01-31T16:25:29,945   copying lm_eval/tasks/blimp/_template_yaml -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:29,947   copying lm_eval/tasks/blimp/README.md -> build/lib/lm_eval/tasks/blimp
2024-01-31T16:25:29,950   copying lm_eval/tasks/mathqa/README.md -> build/lib/lm_eval/tasks/mathqa
2024-01-31T16:25:29,952   copying lm_eval/tasks/hellaswag/README.md -> build/lib/lm_eval/tasks/hellaswag
2024-01-31T16:25:29,955   copying lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/lib/lm_eval/tasks/csatqa
2024-01-31T16:25:29,958   copying lm_eval/tasks/lambada/README.md -> build/lib/lm_eval/tasks/lambada
2024-01-31T16:25:29,960   copying lm_eval/tasks/wikitext/README.md -> build/lib/lm_eval/tasks/wikitext
2024-01-31T16:25:29,962   copying lm_eval/tasks/arithmetic/README.md -> build/lib/lm_eval/tasks/arithmetic
2024-01-31T16:25:29,965   copying lm_eval/tasks/belebele/_default_template_yaml -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:29,967   copying lm_eval/tasks/belebele/README.md -> build/lib/lm_eval/tasks/belebele
2024-01-31T16:25:29,969   copying lm_eval/tasks/kobest/README.md -> build/lib/lm_eval/tasks/kobest
2024-01-31T16:25:29,972   copying lm_eval/tasks/squadv2/README.md -> build/lib/lm_eval/tasks/squadv2
2024-01-31T16:25:29,975   copying lm_eval/tasks/coqa/README.md -> build/lib/lm_eval/tasks/coqa
2024-01-31T16:25:29,978   copying lm_eval/tasks/toxigen/README.md -> build/lib/lm_eval/tasks/toxigen
2024-01-31T16:25:29,980   copying lm_eval/tasks/nq_open/README.md -> build/lib/lm_eval/tasks/nq_open
2024-01-31T16:25:29,981   copying lm_eval/tasks/gsm8k/README.md -> build/lib/lm_eval/tasks/gsm8k
2024-01-31T16:25:29,984   copying lm_eval/tasks/asdiv/README.md -> build/lib/lm_eval/tasks/asdiv
2024-01-31T16:25:29,986   copying lm_eval/tasks/mgsm/README.md -> build/lib/lm_eval/tasks/mgsm
2024-01-31T16:25:29,988   copying lm_eval/tasks/mgsm/direct/direct_yaml -> build/lib/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:29,990   copying lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:29,992   copying lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:29,994   copying lm_eval/tasks/sciq/README.md -> build/lib/lm_eval/tasks/sciq
2024-01-31T16:25:29,996   copying lm_eval/tasks/lambada_multilingual/README.md -> build/lib/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:29,998   copying lm_eval/tasks/piqa/README.md -> build/lib/lm_eval/tasks/piqa
2024-01-31T16:25:30,000   copying lm_eval/tasks/minerva_math/README.md -> build/lib/lm_eval/tasks/minerva_math
2024-01-31T16:25:30,002   copying lm_eval/tasks/scrolls/README.md -> build/lib/lm_eval/tasks/scrolls
2024-01-31T16:25:30,005   copying lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:30,007   copying lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:30,009   copying lm_eval/tasks/logiqa2/README.md -> build/lib/lm_eval/tasks/logiqa2
2024-01-31T16:25:30,012   copying lm_eval/tasks/siqa/README.md -> build/lib/lm_eval/tasks/siqa
2024-01-31T16:25:30,014   copying lm_eval/tasks/fld/README.md -> build/lib/lm_eval/tasks/fld
2024-01-31T16:25:30,016   copying lm_eval/tasks/glue/README.md -> build/lib/lm_eval/tasks/glue
2024-01-31T16:25:30,019   copying lm_eval/tasks/anli/README.md -> build/lib/lm_eval/tasks/anli
2024-01-31T16:25:30,021   copying lm_eval/tasks/drop/README.md -> build/lib/lm_eval/tasks/drop
2024-01-31T16:25:30,023   copying lm_eval/tasks/headqa/README.md -> build/lib/lm_eval/tasks/headqa
2024-01-31T16:25:30,026   copying lm_eval/tasks/qasper/README.md -> build/lib/lm_eval/tasks/qasper
2024-01-31T16:25:30,029   copying lm_eval/tasks/cmmlu/_default_template_yaml -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:30,031   copying lm_eval/tasks/cmmlu/README.md -> build/lib/lm_eval/tasks/cmmlu
2024-01-31T16:25:30,033   copying lm_eval/tasks/paws-x/pawsx_template_yaml -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:30,035   copying lm_eval/tasks/paws-x/README.md -> build/lib/lm_eval/tasks/paws-x
2024-01-31T16:25:30,038   copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:30,040   copying lm_eval/tasks/mmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu/default
2024-01-31T16:25:30,042   copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:30,044   copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:30,046   copying lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:30,052   copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:30,054   copying lm_eval/tasks/prost/README.md -> build/lib/lm_eval/tasks/prost
2024-01-31T16:25:30,057   copying lm_eval/tasks/bbh/README.md -> build/lib/lm_eval/tasks/bbh
2024-01-31T16:25:30,059   copying lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:30,062   copying lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:30,064   copying lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:30,066   copying lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:30,069   copying lm_eval/tasks/lambada_cloze/README.md -> build/lib/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:30,071   copying lm_eval/tasks/super_glue/README.md -> build/lib/lm_eval/tasks/super_glue
2024-01-31T16:25:30,077   copying lm_eval/tasks/openbookqa/README.md -> build/lib/lm_eval/tasks/openbookqa
2024-01-31T16:25:30,080   copying lm_eval/tasks/qa4mre/README.md -> build/lib/lm_eval/tasks/qa4mre
2024-01-31T16:25:30,082   copying lm_eval/tasks/xstorycloze/README.md -> build/lib/lm_eval/tasks/xstorycloze
2024-01-31T16:25:30,085   copying lm_eval/tasks/kmmlu/_default_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:30,087   copying lm_eval/tasks/kmmlu/README.md -> build/lib/lm_eval/tasks/kmmlu
2024-01-31T16:25:30,090   copying lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:30,092   copying lm_eval/tasks/xwinograd/README.md -> build/lib/lm_eval/tasks/xwinograd
2024-01-31T16:25:30,095   copying lm_eval/tasks/xnli/README.md -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:30,097   copying lm_eval/tasks/xnli/xnli_common_yaml -> build/lib/lm_eval/tasks/xnli
2024-01-31T16:25:30,100   copying lm_eval/tasks/unscramble/README.md -> build/lib/lm_eval/tasks/unscramble
2024-01-31T16:25:30,102   copying lm_eval/tasks/mc_taco/README.md -> build/lib/lm_eval/tasks/mc_taco
2024-01-31T16:25:30,105   copying lm_eval/tasks/xcopa/README.md -> build/lib/lm_eval/tasks/xcopa
2024-01-31T16:25:30,108   copying lm_eval/tasks/wsc273/README.md -> build/lib/lm_eval/tasks/wsc273
2024-01-31T16:25:30,110   copying lm_eval/tasks/race/README.md -> build/lib/lm_eval/tasks/race
2024-01-31T16:25:30,113   copying lm_eval/tasks/logiqa/README.md -> build/lib/lm_eval/tasks/logiqa
2024-01-31T16:25:30,765   installing to build/bdist.linux-armv7l/wheel
2024-01-31T16:25:30,765   running install
2024-01-31T16:25:30,790   running install_lib
2024-01-31T16:25:30,794   creating build/bdist.linux-armv7l
2024-01-31T16:25:30,795   creating build/bdist.linux-armv7l/wheel
2024-01-31T16:25:30,797   creating build/bdist.linux-armv7l/wheel/lm_eval
2024-01-31T16:25:30,800   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks
2024-01-31T16:25:30,802   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,803   copying build/lib/lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,805   copying build/lib/lm_eval/tasks/hendrycks_ethics/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,808   copying build/lib/lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,810   copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,812   copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,814   copying build/lib/lm_eval/tasks/hendrycks_ethics/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,816   copying build/lib/lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,818   copying build/lib/lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics
2024-01-31T16:25:30,821   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa
2024-01-31T16:25:30,822   copying build/lib/lm_eval/tasks/medmcqa/medmcqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa
2024-01-31T16:25:30,824   copying build/lib/lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa
2024-01-31T16:25:30,826   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,827   copying build/lib/lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,828   copying build/lib/lm_eval/tasks/pile/pile_gutenberg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,830   copying build/lib/lm_eval/tasks/pile/pile_pile-cc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,832   copying build/lib/lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,833   copying build/lib/lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,835   copying build/lib/lm_eval/tasks/pile/pile_github.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,836   copying build/lib/lm_eval/tasks/pile/pile_hackernews.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,838   copying build/lib/lm_eval/tasks/pile/pile_books3.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,840   copying build/lib/lm_eval/tasks/pile/pile_uspto.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,841   copying build/lib/lm_eval/tasks/pile/pile_arxiv.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,843   copying build/lib/lm_eval/tasks/pile/pile_wikipedia.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,845   copying build/lib/lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,847   copying build/lib/lm_eval/tasks/pile/pile_freelaw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,849   copying build/lib/lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,850   copying build/lib/lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,852   copying build/lib/lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,854   copying build/lib/lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,856   copying build/lib/lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,858   copying build/lib/lm_eval/tasks/pile/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,859   copying build/lib/lm_eval/tasks/pile/pile_europarl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,861   copying build/lib/lm_eval/tasks/pile/pile_enron.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,863   copying build/lib/lm_eval/tasks/pile/pile_stackexchange.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,864   copying build/lib/lm_eval/tasks/pile/pile_philpapers.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile
2024-01-31T16:25:30,866   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa
2024-01-31T16:25:30,867   copying build/lib/lm_eval/tasks/truthfulqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa
2024-01-31T16:25:30,870   copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa
2024-01-31T16:25:30,872   copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa
2024-01-31T16:25:30,873   copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa
2024-01-31T16:25:30,875   copying build/lib/lm_eval/tasks/truthfulqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa
2024-01-31T16:25:30,877   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual
2024-01-31T16:25:30,878   copying build/lib/lm_eval/tasks/mutual/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual
2024-01-31T16:25:30,880   copying build/lib/lm_eval/tasks/mutual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual
2024-01-31T16:25:30,882   copying build/lib/lm_eval/tasks/mutual/multual_plus.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual
2024-01-31T16:25:30,883   copying build/lib/lm_eval/tasks/mutual/mutual.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual
2024-01-31T16:25:30,885   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag
2024-01-31T16:25:30,886   copying build/lib/lm_eval/tasks/swag/swag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag
2024-01-31T16:25:30,888   copying build/lib/lm_eval/tasks/swag/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag
2024-01-31T16:25:30,890   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,891   copying build/lib/lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,893   copying build/lib/lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,895   copying build/lib/lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,896   copying build/lib/lm_eval/tasks/ceval/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,898   copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,900   copying build/lib/lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,902   copying build/lib/lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,904   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,905   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,907   copying build/lib/lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,909   copying build/lib/lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,910   copying build/lib/lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,912   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,914   copying build/lib/lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,915   copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,917   copying build/lib/lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,919   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,920   copying build/lib/lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,922   copying build/lib/lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,924   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,926   copying build/lib/lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,927   copying build/lib/lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,929   copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,930   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,932   copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,934   copying build/lib/lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,935   copying build/lib/lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,937   copying build/lib/lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,939   copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,940   copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,942   copying build/lib/lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,943   copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,945   copying build/lib/lm_eval/tasks/ceval/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,947   copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,949   copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,951   copying build/lib/lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,952   copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,954   copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,955   copying build/lib/lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,957   copying build/lib/lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,959   copying build/lib/lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,961   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,962   copying build/lib/lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,964   copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,966   copying build/lib/lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,967   copying build/lib/lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,969   copying build/lib/lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,971   copying build/lib/lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,973   copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,974   copying build/lib/lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,976   copying build/lib/lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,977   copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,979   copying build/lib/lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,981   copying build/lib/lm_eval/tasks/ceval/_default_ceval_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,982   copying build/lib/lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval
2024-01-31T16:25:30,985   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,985   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,987   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,989   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,991   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,992   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,994   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,996   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,997   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:30,999   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,000   copying build/lib/lm_eval/tasks/crows_pairs/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,002   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,004   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,006   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,007   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,009   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,011   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,013   copying build/lib/lm_eval/tasks/crows_pairs/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,015   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,016   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,018   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,020   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,021   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,023   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,025   copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs
2024-01-31T16:25:31,027   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval
2024-01-31T16:25:31,027   copying build/lib/lm_eval/tasks/ifeval/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval
2024-01-31T16:25:31,030   copying build/lib/lm_eval/tasks/ifeval/instructions_registry.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval
2024-01-31T16:25:31,032   copying build/lib/lm_eval/tasks/ifeval/ifeval.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval
2024-01-31T16:25:31,033   copying build/lib/lm_eval/tasks/ifeval/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval
2024-01-31T16:25:31,035   copying build/lib/lm_eval/tasks/ifeval/instructions.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval
2024-01-31T16:25:31,038   copying build/lib/lm_eval/tasks/ifeval/instructions_util.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval
2024-01-31T16:25:31,041   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze
2024-01-31T16:25:31,042   copying build/lib/lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze
2024-01-31T16:25:31,044   copying build/lib/lm_eval/tasks/storycloze/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze
2024-01-31T16:25:31,046   copying build/lib/lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze
2024-01-31T16:25:31,048   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa
2024-01-31T16:25:31,049   copying build/lib/lm_eval/tasks/pubmedqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa
2024-01-31T16:25:31,050   copying build/lib/lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa
2024-01-31T16:25:31,052   copying build/lib/lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa
2024-01-31T16:25:31,054   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016
2024-01-31T16:25:31,055   copying build/lib/lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016
2024-01-31T16:25:31,057   copying build/lib/lm_eval/tasks/wmt2016/metrics.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016
2024-01-31T16:25:31,059   copying build/lib/lm_eval/tasks/wmt2016/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016
2024-01-31T16:25:31,060   copying build/lib/lm_eval/tasks/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks
2024-01-31T16:25:31,063   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals
2024-01-31T16:25:31,064   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/winogenerated
2024-01-31T16:25:31,066   copying build/lib/lm_eval/tasks/model_written_evals/winogenerated/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/winogenerated
2024-01-31T16:25:31,070   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,071   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,073   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,075   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,076   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,078   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,080   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,081   copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,083   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,084   copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,086   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,088   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,089   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,091   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,093   copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,094   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,096   copying build/lib/lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,098   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,099   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,101   copying build/lib/lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,103   copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,105   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,106   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,108   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,110   copying build/lib/lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,111   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,113   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,114   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,116   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,118   copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,119   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,121   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,123   copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,124   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,126   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,128   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,130   copying build/lib/lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,131   copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,133   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,135   copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,136   copying build/lib/lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,138   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,140   copying build/lib/lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,141   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,143   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,144   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,146   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,148   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,149   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,151   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,153   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,154   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,156   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,158   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,160   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,161   copying build/lib/lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,163   copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,165   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,166   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,168   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,170   copying build/lib/lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,172   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,177   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,179   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,180   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,182   copying build/lib/lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,184   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,185   copying build/lib/lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,187   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,188   copying build/lib/lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,190   copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,192   copying build/lib/lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,193   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,195   copying build/lib/lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,197   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,198   copying build/lib/lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,200   copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,202   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,204   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,205   copying build/lib/lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,207   copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,209   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,211   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,212   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,214   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,215   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,217   copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,219   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,220   copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,222   copying build/lib/lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,224   copying build/lib/lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,225   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,227   copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,229   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,230   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,232   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,234   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,235   copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,237   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,239   copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,240   copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,242   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,244   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,245   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,247   copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,249   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,250   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,252   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,254   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,255   copying build/lib/lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,257   copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,259   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,260   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,262   copying build/lib/lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,264   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,265   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,267   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,269   copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,271   copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,272   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,274   copying build/lib/lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,276   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,278   copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,279   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,281   copying build/lib/lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,283   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,284   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,286   copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,288   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,289   copying build/lib/lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,291   copying build/lib/lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,293   copying build/lib/lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,294   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,296   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,298   copying build/lib/lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,300   copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,301   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,303   copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona
2024-01-31T16:25:31,306   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,307   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,309   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,311   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,312   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,314   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,316   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,317   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,319   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,321   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,322   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,324   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,325   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,327   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,329   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,331   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,332   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,334   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,336   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,337   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,339   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,341   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,343   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,344   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,346   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,348   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,349   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,351   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,353   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,354   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,356   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,358   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,360   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,361   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,363   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,365   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,366   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,368   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,370   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,372   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,373   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,375   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,377   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,378   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,380   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,382   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,383   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,385   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,387   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,388   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,390   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,392   copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk
2024-01-31T16:25:31,394   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:31,395   copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:31,396   copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:31,398   copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy
2024-01-31T16:25:31,400   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,401   copying build/lib/lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,403   copying build/lib/lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,405   copying build/lib/lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,407   copying build/lib/lm_eval/tasks/translation/wmt16_en-de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,408   copying build/lib/lm_eval/tasks/translation/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,411   copying build/lib/lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,412   copying build/lib/lm_eval/tasks/translation/wmt16_de-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,414   copying build/lib/lm_eval/tasks/translation/wmt_common_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,416   copying build/lib/lm_eval/tasks/translation/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,417   copying build/lib/lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,419   copying build/lib/lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation
2024-01-31T16:25:31,421   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande
2024-01-31T16:25:31,422   copying build/lib/lm_eval/tasks/winogrande/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande
2024-01-31T16:25:31,424   copying build/lib/lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande
2024-01-31T16:25:31,425   copying build/lib/lm_eval/tasks/winogrande/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande
2024-01-31T16:25:31,428   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench
2024-01-31T16:25:31,429   copying build/lib/lm_eval/tasks/bigbench/generate_tasks.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench
2024-01-31T16:25:31,431   copying build/lib/lm_eval/tasks/bigbench/generate_until_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench
2024-01-31T16:25:31,435   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,436   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,438   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,440   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,441   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,443   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/gem.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,445   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,447   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,449   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,450   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,452   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,454   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,455   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,457   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,459   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,461   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,463   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_transliterate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,465   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,466   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,468   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,469   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,471   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,472   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,474   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/language_games.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,476   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,477   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/linguistics_puzzles.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,479   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,481   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,482   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,484   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,486   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,487   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/minute_mysteries_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,489   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/natural_instructions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,491   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,493   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,494   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,496   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,498   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,500   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,501   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,503   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,505   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,506   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/conlang_translation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,508   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/auto_categorization.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,509   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,511   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,513   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/linguistic_mappings.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,514   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,516   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,518   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,519   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/list_functions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,521   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,523   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/gender_inclusive_sentences_german.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,524   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,526   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,528   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,530   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,531   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,533   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,535   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,537   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,538   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,540   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,542   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,543   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,545   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,547   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,548   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,550   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,551   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,553   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,555   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/matrixshapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,557   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mult_data_wrangling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,558   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/chess_state_tracking.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,560   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,562   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/word_unscrambling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,564   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,566   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,567   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,569   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,571   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cryptonite.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,573   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,574   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,576   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,578   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,580   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_text_editing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,581   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,583   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,584   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,586   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,588   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/scientific_press_release.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,589   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,591   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,592   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,594   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,596   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,598   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/disfl_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,600   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,601   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,603   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,605   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,606   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,608   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,610   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,611   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,613   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,615   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sufficient_information.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,617   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,618   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,620   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,621   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_in_context_sparc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,623   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,625   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,626   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,628   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,630   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,631   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,633   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,635   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,637   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/modified_arithmetic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,638   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,640   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,642   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,643   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/bridging_anaphora_resolution_barqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,645   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,647   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,649   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,650   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,652   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,654   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,655   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,657   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,658   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,660   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,662   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/few_shot_nlg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,663   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,665   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,666   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,668   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,670   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,671   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/ascii_word_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,673   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,675   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/operators.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,676   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,678   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/tense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,680   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,681   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,683   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,685   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,686   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,688   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,690   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/codenames.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,692   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_spider.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,693   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,695   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,696   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,698   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,700   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,701   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,703   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/chinese_remainder_theorem.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,704   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,706   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,708   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,709   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,711   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,713   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,714   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hindi_question_answering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,716   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,718   copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice
2024-01-31T16:25:31,720   copying build/lib/lm_eval/tasks/bigbench/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench
2024-01-31T16:25:31,725   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,726   copying build/lib/lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,728   copying build/lib/lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,729   copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,731   copying build/lib/lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,733   copying build/lib/lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,735   copying build/lib/lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,737   copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,738   copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,740   copying build/lib/lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,742   copying build/lib/lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,743   copying build/lib/lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,745   copying build/lib/lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,746   copying build/lib/lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,748   copying build/lib/lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,749   copying build/lib/lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,751   copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,753   copying build/lib/lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,754   copying build/lib/lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,756   copying build/lib/lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,758   copying build/lib/lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,760   copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,761   copying build/lib/lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,763   copying build/lib/lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,765   copying build/lib/lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,766   copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,768   copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,770   copying build/lib/lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,772   copying build/lib/lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,773   copying build/lib/lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,775   copying build/lib/lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,776   copying build/lib/lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,778   copying build/lib/lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,780   copying build/lib/lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,781   copying build/lib/lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,783   copying build/lib/lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,785   copying build/lib/lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,786   copying build/lib/lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,788   copying build/lib/lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,790   copying build/lib/lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,791   copying build/lib/lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,793   copying build/lib/lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,795   copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,797   copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,798   copying build/lib/lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,800   copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,801   copying build/lib/lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,803   copying build/lib/lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,805   copying build/lib/lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,807   copying build/lib/lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,808   copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,810   copying build/lib/lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,812   copying build/lib/lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,813   copying build/lib/lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,815   copying build/lib/lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,816   copying build/lib/lm_eval/tasks/bigbench/generate_until/color.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,818   copying build/lib/lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,819   copying build/lib/lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,821   copying build/lib/lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,823   copying build/lib/lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,824   copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,826   copying build/lib/lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,828   copying build/lib/lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,829   copying build/lib/lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,831   copying build/lib/lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,833   copying build/lib/lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,834   copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,836   copying build/lib/lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,838   copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,839   copying build/lib/lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,841   copying build/lib/lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,843   copying build/lib/lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,844   copying build/lib/lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,846   copying build/lib/lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,848   copying build/lib/lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,850   copying build/lib/lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,851   copying build/lib/lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,853   copying build/lib/lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,854   copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,856   copying build/lib/lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,857   copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,859   copying build/lib/lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,860   copying build/lib/lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,862   copying build/lib/lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,864   copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,865   copying build/lib/lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,867   copying build/lib/lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,869   copying build/lib/lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,870   copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,872   copying build/lib/lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,874   copying build/lib/lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,875   copying build/lib/lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,878   copying build/lib/lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,879   copying build/lib/lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,881   copying build/lib/lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,883   copying build/lib/lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,884   copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,886   copying build/lib/lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,888   copying build/lib/lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,890   copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,891   copying build/lib/lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,893   copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,895   copying build/lib/lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,896   copying build/lib/lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,898   copying build/lib/lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,899   copying build/lib/lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,901   copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,903   copying build/lib/lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,904   copying build/lib/lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,906   copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,908   copying build/lib/lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,909   copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,911   copying build/lib/lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,913   copying build/lib/lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,914   copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,916   copying build/lib/lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,917   copying build/lib/lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,919   copying build/lib/lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,921   copying build/lib/lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,923   copying build/lib/lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,924   copying build/lib/lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,926   copying build/lib/lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,928   copying build/lib/lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,929   copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,931   copying build/lib/lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,933   copying build/lib/lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,934   copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,936   copying build/lib/lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,938   copying build/lib/lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,939   copying build/lib/lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,941   copying build/lib/lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,943   copying build/lib/lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,944   copying build/lib/lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,946   copying build/lib/lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,948   copying build/lib/lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,950   copying build/lib/lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,951   copying build/lib/lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,953   copying build/lib/lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,955   copying build/lib/lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,956   copying build/lib/lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,958   copying build/lib/lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,960   copying build/lib/lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,961   copying build/lib/lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,963   copying build/lib/lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,965   copying build/lib/lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,966   copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,968   copying build/lib/lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,970   copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,971   copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,973   copying build/lib/lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,974   copying build/lib/lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,976   copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,977   copying build/lib/lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,979   copying build/lib/lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,981   copying build/lib/lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,983   copying build/lib/lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,984   copying build/lib/lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,986   copying build/lib/lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,988   copying build/lib/lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,989   copying build/lib/lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,991   copying build/lib/lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,993   copying build/lib/lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,994   copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,996   copying build/lib/lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,998   copying build/lib/lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:31,999   copying build/lib/lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:32,001   copying build/lib/lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:32,003   copying build/lib/lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until
2024-01-31T16:25:32,004   copying build/lib/lm_eval/tasks/bigbench/multiple_choice_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench
2024-01-31T16:25:32,006   copying build/lib/lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench
2024-01-31T16:25:32,008   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa
2024-01-31T16:25:32,009   copying build/lib/lm_eval/tasks/triviaqa/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa
2024-01-31T16:25:32,010   copying build/lib/lm_eval/tasks/triviaqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa
2024-01-31T16:25:32,012   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2
2024-01-31T16:25:32,013   copying build/lib/lm_eval/tasks/polemo2/polemo2_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2
2024-01-31T16:25:32,015   copying build/lib/lm_eval/tasks/polemo2/polemo2_in.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2
2024-01-31T16:25:32,016   copying build/lib/lm_eval/tasks/polemo2/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2
2024-01-31T16:25:32,019   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs
2024-01-31T16:25:32,019   copying build/lib/lm_eval/tasks/webqs/webqs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs
2024-01-31T16:25:32,021   copying build/lib/lm_eval/tasks/webqs/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs
2024-01-31T16:25:32,023   copying build/lib/lm_eval/tasks/webqs/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs
2024-01-31T16:25:32,025   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks
2024-01-31T16:25:32,026   copying build/lib/lm_eval/tasks/benchmarks/pythia.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks
2024-01-31T16:25:32,028   copying build/lib/lm_eval/tasks/benchmarks/minerva_math.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks
2024-01-31T16:25:32,030   copying build/lib/lm_eval/tasks/benchmarks/t0_eval.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks
2024-01-31T16:25:32,032   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa
2024-01-31T16:25:32,033   copying build/lib/lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa
2024-01-31T16:25:32,035   copying build/lib/lm_eval/tasks/benchmarks/multimedqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa
2024-01-31T16:25:32,037   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,038   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,040   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_rte.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,042   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_anli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,043   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_boolq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,045   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_arc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,047   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,049   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/yaml_templates
2024-01-31T16:25:32,050   copying build/lib/lm_eval/tasks/benchmarks/flan/yaml_templates/cot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/yaml_templates
2024-01-31T16:25:32,052   copying build/lib/lm_eval/tasks/benchmarks/flan/yaml_templates/held_in_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/yaml_templates
2024-01-31T16:25:32,054   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_in_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,056   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:32,057   copying build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates/arc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:32,058   copying build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates/anli.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:32,060   copying build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates/boolq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:32,062   copying build/lib/lm_eval/tasks/benchmarks/flan/prompt_templates/rte.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan/prompt_templates
2024-01-31T16:25:32,064   copying build/lib/lm_eval/tasks/benchmarks/flan/flan_cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan
2024-01-31T16:25:32,066   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi
2024-01-31T16:25:32,066   copying build/lib/lm_eval/tasks/babi/babi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi
2024-01-31T16:25:32,068   copying build/lib/lm_eval/tasks/babi/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi
2024-01-31T16:25:32,070   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc
2024-01-31T16:25:32,071   copying build/lib/lm_eval/tasks/arc/arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc
2024-01-31T16:25:32,073   copying build/lib/lm_eval/tasks/arc/arc_easy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc
2024-01-31T16:25:32,075   copying build/lib/lm_eval/tasks/arc/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc
2024-01-31T16:25:32,078   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,079   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,081   copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,082   copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,084   copying build/lib/lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,086   copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,087   copying build/lib/lm_eval/tasks/blimp/npi_present_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,089   copying build/lib/lm_eval/tasks/blimp/npi_present_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,091   copying build/lib/lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,093   copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,094   copying build/lib/lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,096   copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,098   copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,100   copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,101   copying build/lib/lm_eval/tasks/blimp/causative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,103   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,104   copying build/lib/lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,106   copying build/lib/lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,107   copying build/lib/lm_eval/tasks/blimp/wh_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,109   copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,110   copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,112   copying build/lib/lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,114   copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,115   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,117   copying build/lib/lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,119   copying build/lib/lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,121   copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,122   copying build/lib/lm_eval/tasks/blimp/inchoative.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,124   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,126   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,127   copying build/lib/lm_eval/tasks/blimp/drop_argument.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,129   copying build/lib/lm_eval/tasks/blimp/_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,131   copying build/lib/lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,133   copying build/lib/lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,136   copying build/lib/lm_eval/tasks/blimp/passive_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,138   copying build/lib/lm_eval/tasks/blimp/generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,140   copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,141   copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,143   copying build/lib/lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,145   copying build/lib/lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,146   copying build/lib/lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,148   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,150   copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,151   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,153   copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,154   copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,156   copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,158   copying build/lib/lm_eval/tasks/blimp/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,159   copying build/lib/lm_eval/tasks/blimp/complex_NP_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,161   copying build/lib/lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,163   copying build/lib/lm_eval/tasks/blimp/passive_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,164   copying build/lib/lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,166   copying build/lib/lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,167   copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,169   copying build/lib/lm_eval/tasks/blimp/transitive.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,171   copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,173   copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,174   copying build/lib/lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,176   copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,178   copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,179   copying build/lib/lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,181   copying build/lib/lm_eval/tasks/blimp/adjunct_island.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,183   copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,185   copying build/lib/lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,186   copying build/lib/lm_eval/tasks/blimp/only_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,188   copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,190   copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,191   copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,193   copying build/lib/lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,195   copying build/lib/lm_eval/tasks/blimp/intransitive.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,196   copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp
2024-01-31T16:25:32,198   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa
2024-01-31T16:25:32,199   copying build/lib/lm_eval/tasks/mathqa/mathqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa
2024-01-31T16:25:32,201   copying build/lib/lm_eval/tasks/mathqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa
2024-01-31T16:25:32,202   copying build/lib/lm_eval/tasks/mathqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa
2024-01-31T16:25:32,205   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag
2024-01-31T16:25:32,205   copying build/lib/lm_eval/tasks/hellaswag/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag
2024-01-31T16:25:32,207   copying build/lib/lm_eval/tasks/hellaswag/hellaswag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag
2024-01-31T16:25:32,209   copying build/lib/lm_eval/tasks/hellaswag/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag
2024-01-31T16:25:32,211   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,212   copying build/lib/lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,214   copying build/lib/lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,215   copying build/lib/lm_eval/tasks/csatqa/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,217   copying build/lib/lm_eval/tasks/csatqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,219   copying build/lib/lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,221   copying build/lib/lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,222   copying build/lib/lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,224   copying build/lib/lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,226   copying build/lib/lm_eval/tasks/csatqa/csatqa_li.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa
2024-01-31T16:25:32,228   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada
2024-01-31T16:25:32,229   copying build/lib/lm_eval/tasks/lambada/lambada_openai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada
2024-01-31T16:25:32,231   copying build/lib/lm_eval/tasks/lambada/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada
2024-01-31T16:25:32,233   copying build/lib/lm_eval/tasks/lambada/lambada_standard.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada
2024-01-31T16:25:32,235   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext
2024-01-31T16:25:32,236   copying build/lib/lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext
2024-01-31T16:25:32,237   copying build/lib/lm_eval/tasks/wikitext/wikitext.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext
2024-01-31T16:25:32,239   copying build/lib/lm_eval/tasks/wikitext/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext
2024-01-31T16:25:32,241   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,242   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,244   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,246   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,247   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,249   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,251   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,252   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,254   copying build/lib/lm_eval/tasks/arithmetic/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,256   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,257   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,259   copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic
2024-01-31T16:25:32,263   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,263   copying build/lib/lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,265   copying build/lib/lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,267   copying build/lib/lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,269   copying build/lib/lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,270   copying build/lib/lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,272   copying build/lib/lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,274   copying build/lib/lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,275   copying build/lib/lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,277   copying build/lib/lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,279   copying build/lib/lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,281   copying build/lib/lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,282   copying build/lib/lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,284   copying build/lib/lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,286   copying build/lib/lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,287   copying build/lib/lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,289   copying build/lib/lm_eval/tasks/belebele/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,291   copying build/lib/lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,292   copying build/lib/lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,294   copying build/lib/lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,296   copying build/lib/lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,297   copying build/lib/lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,299   copying build/lib/lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,300   copying build/lib/lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,302   copying build/lib/lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,303   copying build/lib/lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,305   copying build/lib/lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,307   copying build/lib/lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,308   copying build/lib/lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,310   copying build/lib/lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,311   copying build/lib/lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,313   copying build/lib/lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,315   copying build/lib/lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,316   copying build/lib/lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,318   copying build/lib/lm_eval/tasks/belebele/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,320   copying build/lib/lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,321   copying build/lib/lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,323   copying build/lib/lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,325   copying build/lib/lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,326   copying build/lib/lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,328   copying build/lib/lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,330   copying build/lib/lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,331   copying build/lib/lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,333   copying build/lib/lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,335   copying build/lib/lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,336   copying build/lib/lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,338   copying build/lib/lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,340   copying build/lib/lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,341   copying build/lib/lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,343   copying build/lib/lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,344   copying build/lib/lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,346   copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,347   copying build/lib/lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,349   copying build/lib/lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,351   copying build/lib/lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,352   copying build/lib/lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,354   copying build/lib/lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,356   copying build/lib/lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,357   copying build/lib/lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,359   copying build/lib/lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,361   copying build/lib/lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,362   copying build/lib/lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,364   copying build/lib/lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,366   copying build/lib/lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,367   copying build/lib/lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,369   copying build/lib/lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,371   copying build/lib/lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,372   copying build/lib/lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,374   copying build/lib/lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,376   copying build/lib/lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,378   copying build/lib/lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,379   copying build/lib/lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,381   copying build/lib/lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,383   copying build/lib/lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,384   copying build/lib/lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,386   copying build/lib/lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,388   copying build/lib/lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,390   copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,391   copying build/lib/lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,393   copying build/lib/lm_eval/tasks/belebele/belebele_default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,394   copying build/lib/lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,396   copying build/lib/lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,398   copying build/lib/lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,399   copying build/lib/lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,401   copying build/lib/lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,403   copying build/lib/lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,404   copying build/lib/lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,406   copying build/lib/lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,408   copying build/lib/lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,410   copying build/lib/lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,411   copying build/lib/lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,413   copying build/lib/lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,415   copying build/lib/lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,416   copying build/lib/lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,418   copying build/lib/lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,420   copying build/lib/lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,421   copying build/lib/lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,423   copying build/lib/lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,425   copying build/lib/lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,426   copying build/lib/lm_eval/tasks/belebele/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,428   copying build/lib/lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,430   copying build/lib/lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,432   copying build/lib/lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,434   copying build/lib/lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,435   copying build/lib/lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,437   copying build/lib/lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,438   copying build/lib/lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,440   copying build/lib/lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,441   copying build/lib/lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,443   copying build/lib/lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,444   copying build/lib/lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,446   copying build/lib/lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,447   copying build/lib/lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,449   copying build/lib/lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,451   copying build/lib/lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,452   copying build/lib/lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,454   copying build/lib/lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,456   copying build/lib/lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,457   copying build/lib/lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,459   copying build/lib/lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,461   copying build/lib/lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,462   copying build/lib/lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,464   copying build/lib/lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,466   copying build/lib/lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,467   copying build/lib/lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,469   copying build/lib/lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,471   copying build/lib/lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele
2024-01-31T16:25:32,473   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,474   copying build/lib/lm_eval/tasks/kobest/kobest_boolq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,476   copying build/lib/lm_eval/tasks/kobest/kobest_copa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,477   copying build/lib/lm_eval/tasks/kobest/kobest_wic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,479   copying build/lib/lm_eval/tasks/kobest/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,481   copying build/lib/lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,482   copying build/lib/lm_eval/tasks/kobest/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,484   copying build/lib/lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest
2024-01-31T16:25:32,486   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue
2024-01-31T16:25:32,487   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,488   copying build/lib/lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,490   copying build/lib/lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,492   copying build/lib/lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,493   copying build/lib/lm_eval/tasks/code_x_glue/code-text/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,495   copying build/lib/lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,497   copying build/lib/lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,498   copying build/lib/lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,500   copying build/lib/lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text
2024-01-31T16:25:32,503   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2
2024-01-31T16:25:32,504   copying build/lib/lm_eval/tasks/squadv2/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2
2024-01-31T16:25:32,506   copying build/lib/lm_eval/tasks/squadv2/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2
2024-01-31T16:25:32,508   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa
2024-01-31T16:25:32,509   copying build/lib/lm_eval/tasks/coqa/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa
2024-01-31T16:25:32,511   copying build/lib/lm_eval/tasks/coqa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa
2024-01-31T16:25:32,513   copying build/lib/lm_eval/tasks/coqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa
2024-01-31T16:25:32,515   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen
2024-01-31T16:25:32,516   copying build/lib/lm_eval/tasks/toxigen/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen
2024-01-31T16:25:32,518   copying build/lib/lm_eval/tasks/toxigen/toxigen.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen
2024-01-31T16:25:32,519   copying build/lib/lm_eval/tasks/toxigen/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen
2024-01-31T16:25:32,521   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open
2024-01-31T16:25:32,522   copying build/lib/lm_eval/tasks/nq_open/nq_open.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open
2024-01-31T16:25:32,524   copying build/lib/lm_eval/tasks/nq_open/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open
2024-01-31T16:25:32,526   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k
2024-01-31T16:25:32,527   copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k
2024-01-31T16:25:32,529   copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k
2024-01-31T16:25:32,531   copying build/lib/lm_eval/tasks/gsm8k/gsm8k.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k
2024-01-31T16:25:32,533   copying build/lib/lm_eval/tasks/gsm8k/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k
2024-01-31T16:25:32,535   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv
2024-01-31T16:25:32,535   copying build/lib/lm_eval/tasks/asdiv/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv
2024-01-31T16:25:32,537   copying build/lib/lm_eval/tasks/asdiv/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv
2024-01-31T16:25:32,539   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm
2024-01-31T16:25:32,540   copying build/lib/lm_eval/tasks/mgsm/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm
2024-01-31T16:25:32,543   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,544   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,546   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,547   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,549   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,550   copying build/lib/lm_eval/tasks/mgsm/direct/direct_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,552   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,554   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,556   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,557   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,559   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,561   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,562   copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct
2024-01-31T16:25:32,564   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,565   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_te_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,567   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_sw_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,569   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_ru_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,570   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_de_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,572   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_es_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,574   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,575   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_zh_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,577   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_fr_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,579   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_bn_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,581   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_ja_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,582   copying build/lib/lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,584   copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_th_en-cot.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot
2024-01-31T16:25:32,586   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,587   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,589   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,590   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,592   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,593   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,595   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,596   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,598   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_ja.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,600   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,602   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,603   copying build/lib/lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,605   copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot
2024-01-31T16:25:32,607   copying build/lib/lm_eval/tasks/mgsm/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm
2024-01-31T16:25:32,609   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq
2024-01-31T16:25:32,610   copying build/lib/lm_eval/tasks/sciq/sciq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq
2024-01-31T16:25:32,611   copying build/lib/lm_eval/tasks/sciq/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq
2024-01-31T16:25:32,614   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:32,615   copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:32,616   copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:32,618   copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:32,620   copying build/lib/lm_eval/tasks/lambada_multilingual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:32,622   copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:32,623   copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual
2024-01-31T16:25:32,625   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa
2024-01-31T16:25:32,626   copying build/lib/lm_eval/tasks/piqa/piqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa
2024-01-31T16:25:32,628   copying build/lib/lm_eval/tasks/piqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa
2024-01-31T16:25:32,630   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,631   copying build/lib/lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,633   copying build/lib/lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,634   copying build/lib/lm_eval/tasks/minerva_math/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,637   copying build/lib/lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,638   copying build/lib/lm_eval/tasks/minerva_math/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,640   copying build/lib/lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,641   copying build/lib/lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,643   copying build/lib/lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,645   copying build/lib/lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math
2024-01-31T16:25:32,647   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls
2024-01-31T16:25:32,648   copying build/lib/lm_eval/tasks/scrolls/scrolls.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls
2024-01-31T16:25:32,650   copying build/lib/lm_eval/tasks/scrolls/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls
2024-01-31T16:25:32,651   copying build/lib/lm_eval/tasks/scrolls/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls
2024-01-31T16:25:32,654   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi
2024-01-31T16:25:32,656   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,657   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,659   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,661   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,663   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,664   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,666   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,668   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,669   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,671   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,673   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,675   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,676   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,678   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,679   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,681   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,682   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,684   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,686   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,687   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,689   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,691   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,692   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,694   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,696   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,698   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,699   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,701   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,703   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,704   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,706   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,708   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,710   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,711   copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual
2024-01-31T16:25:32,713   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2
2024-01-31T16:25:32,714   copying build/lib/lm_eval/tasks/logiqa2/logieval.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2
2024-01-31T16:25:32,716   copying build/lib/lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2
2024-01-31T16:25:32,717   copying build/lib/lm_eval/tasks/logiqa2/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2
2024-01-31T16:25:32,719   copying build/lib/lm_eval/tasks/logiqa2/logiqa2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2
2024-01-31T16:25:32,721   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa
2024-01-31T16:25:32,722   copying build/lib/lm_eval/tasks/siqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa
2024-01-31T16:25:32,724   copying build/lib/lm_eval/tasks/siqa/siqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa
2024-01-31T16:25:32,726   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld
2024-01-31T16:25:32,726   copying build/lib/lm_eval/tasks/fld/fld_star.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld
2024-01-31T16:25:32,728   copying build/lib/lm_eval/tasks/fld/fld_default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld
2024-01-31T16:25:32,730   copying build/lib/lm_eval/tasks/fld/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld
2024-01-31T16:25:32,732   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa
2024-01-31T16:25:32,733   copying build/lib/lm_eval/tasks/medqa/medqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa
2024-01-31T16:25:32,735   copying build/lib/lm_eval/tasks/medqa/preprocess_medqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa
2024-01-31T16:25:32,737   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue
2024-01-31T16:25:32,738   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/sst2
2024-01-31T16:25:32,739   copying build/lib/lm_eval/tasks/glue/sst2/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/sst2
2024-01-31T16:25:32,742   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mrpc
2024-01-31T16:25:32,742   copying build/lib/lm_eval/tasks/glue/mrpc/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mrpc
2024-01-31T16:25:32,745   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/rte
2024-01-31T16:25:32,746   copying build/lib/lm_eval/tasks/glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/rte
2024-01-31T16:25:32,748   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli
2024-01-31T16:25:32,749   copying build/lib/lm_eval/tasks/glue/mnli/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli
2024-01-31T16:25:32,750   copying build/lib/lm_eval/tasks/glue/mnli/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli
2024-01-31T16:25:32,752   copying build/lib/lm_eval/tasks/glue/mnli/mismatch.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli
2024-01-31T16:25:32,754   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/wnli
2024-01-31T16:25:32,755   copying build/lib/lm_eval/tasks/glue/wnli/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/wnli
2024-01-31T16:25:32,757   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qnli
2024-01-31T16:25:32,758   copying build/lib/lm_eval/tasks/glue/qnli/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qnli
2024-01-31T16:25:32,760   copying build/lib/lm_eval/tasks/glue/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue
2024-01-31T16:25:32,762   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qqp
2024-01-31T16:25:32,763   copying build/lib/lm_eval/tasks/glue/qqp/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qqp
2024-01-31T16:25:32,766   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/cola
2024-01-31T16:25:32,766   copying build/lib/lm_eval/tasks/glue/cola/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/cola
2024-01-31T16:25:32,769   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli
2024-01-31T16:25:32,769   copying build/lib/lm_eval/tasks/anli/anli_r1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli
2024-01-31T16:25:32,771   copying build/lib/lm_eval/tasks/anli/anli_r2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli
2024-01-31T16:25:32,773   copying build/lib/lm_eval/tasks/anli/anli_r3.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli
2024-01-31T16:25:32,774   copying build/lib/lm_eval/tasks/anli/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli
2024-01-31T16:25:32,776   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop
2024-01-31T16:25:32,777   copying build/lib/lm_eval/tasks/drop/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop
2024-01-31T16:25:32,779   copying build/lib/lm_eval/tasks/drop/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop
2024-01-31T16:25:32,781   copying build/lib/lm_eval/tasks/drop/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop
2024-01-31T16:25:32,783   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa
2024-01-31T16:25:32,784   copying build/lib/lm_eval/tasks/headqa/headqa_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa
2024-01-31T16:25:32,785   copying build/lib/lm_eval/tasks/headqa/headqa_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa
2024-01-31T16:25:32,787   copying build/lib/lm_eval/tasks/headqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa
2024-01-31T16:25:32,789   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper
2024-01-31T16:25:32,790   copying build/lib/lm_eval/tasks/qasper/bool.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper
2024-01-31T16:25:32,792   copying build/lib/lm_eval/tasks/qasper/metrics.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper
2024-01-31T16:25:32,793   copying build/lib/lm_eval/tasks/qasper/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper
2024-01-31T16:25:32,795   copying build/lib/lm_eval/tasks/qasper/freeform.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper
2024-01-31T16:25:32,797   copying build/lib/lm_eval/tasks/qasper/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper
2024-01-31T16:25:32,800   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,801   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,802   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,804   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_medical_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,806   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_engineering_hydrology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,807   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_civil_service_exam.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,809   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,811   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,812   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,814   copying build/lib/lm_eval/tasks/cmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,816   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_legal_and_moral_basis.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,818   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,820   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_ancient_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,821   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,823   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,825   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_arts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,826   copying build/lib/lm_eval/tasks/cmmlu/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,828   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,829   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,831   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,833   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,834   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_marxist_theory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,836   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,837   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,839   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,841   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_driving_rule.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,842   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,844   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,846   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,848   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,849   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_food_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,851   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,853   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,854   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,856   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,858   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,860   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,861   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,863   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,864   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_ethnology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,866   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,867   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,869   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_food_culture.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,871   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,872   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,874   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_construction_project_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,876   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,877   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,879   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_commonsense.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,881   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,882   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,884   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,886   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,887   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,889   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_college_actuarial_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,891   copying build/lib/lm_eval/tasks/cmmlu/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,893   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_elementary_information_and_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,894   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_journalism.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,896   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,897   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_logical.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,899   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,901   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_agronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,903   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,905   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_chinese_literature.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,906   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,908   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,909   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,911   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_modern_chinese.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,912   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,914   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,915   copying build/lib/lm_eval/tasks/cmmlu/cmmlu_default_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu
2024-01-31T16:25:32,918   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,918   copying build/lib/lm_eval/tasks/paws-x/paws_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,920   copying build/lib/lm_eval/tasks/paws-x/paws_ja.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,922   copying build/lib/lm_eval/tasks/paws-x/pawsx_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,923   copying build/lib/lm_eval/tasks/paws-x/paws_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,925   copying build/lib/lm_eval/tasks/paws-x/paws_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,926   copying build/lib/lm_eval/tasks/paws-x/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,928   copying build/lib/lm_eval/tasks/paws-x/_generate_config.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,930   copying build/lib/lm_eval/tasks/paws-x/paws_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,932   copying build/lib/lm_eval/tasks/paws-x/paws_ko.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,933   copying build/lib/lm_eval/tasks/paws-x/paws_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x
2024-01-31T16:25:32,935   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu
2024-01-31T16:25:32,936   copying build/lib/lm_eval/tasks/mmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu
2024-01-31T16:25:32,940   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,941   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,942   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,944   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,946   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,947   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,949   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,951   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,953   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,954   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,956   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,957   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,959   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,961   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,962   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,964   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,965   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,967   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,969   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,970   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,972   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,973   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,975   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,977   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,978   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,980   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,982   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,984   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,985   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,987   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,989   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,991   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,992   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,994   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,995   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,997   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:32,999   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,000   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,002   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,003   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,005   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,007   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,009   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,010   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,012   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,014   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,015   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,017   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,019   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,021   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,022   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,024   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,026   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,028   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,029   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,031   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,032   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,034   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,035   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,037   copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot
2024-01-31T16:25:33,040   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,041   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,042   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,044   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,046   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,047   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,049   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,051   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,053   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,054   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,056   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,058   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,059   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,061   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,063   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,065   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,066   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,068   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,070   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,071   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,073   copying build/lib/lm_eval/tasks/mmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,075   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,076   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,078   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,080   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,081   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,083   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,085   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,086   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,088   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,090   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,091   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,093   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,095   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,096   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,098   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,100   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,101   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,103   copying build/lib/lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,105   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,106   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,108   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,110   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,111   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,113   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,115   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,116   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,118   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,120   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,121   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,123   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,124   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,126   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,128   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,129   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,131   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,133   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,134   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,136   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,138   copying build/lib/lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default
2024-01-31T16:25:33,140   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot
2024-01-31T16:25:33,142   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,143   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,145   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,146   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,148   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,150   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,151   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,153   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,155   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,161   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,163   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,164   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,166   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,168   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,170   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,171   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,173   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,175   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,177   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,178   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,180   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,182   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,184   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,185   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,187   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,189   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,191   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,193   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,195   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,196   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,198   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,200   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,202   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,204   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,205   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,207   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,209   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,211   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,212   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,214   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,216   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,217   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,219   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,221   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,223   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,224   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,226   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,228   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,229   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,231   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,233   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,234   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,236   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,238   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,239   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,241   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,242   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,244   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,246   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,247   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood
2024-01-31T16:25:33,250   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,251   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,253   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,255   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,257   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,258   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,260   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,262   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,263   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,265   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,267   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,269   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,270   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,272   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,274   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,275   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,277   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,279   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,280   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,282   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,284   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,285   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,287   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,289   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,290   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,292   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,294   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,296   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,297   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,299   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,301   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,302   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,304   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,306   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,308   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,309   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,311   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,312   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,314   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,316   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,317   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,319   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,321   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,322   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,324   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,326   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,327   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,329   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,331   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,333   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,334   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,336   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,338   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,339   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,341   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,343   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,344   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,346   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,347   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,349   copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative
2024-01-31T16:25:33,352   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,352   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,354   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,356   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,358   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,360   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,362   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,364   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,366   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,367   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,369   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,371   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,373   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,378   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,380   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,382   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,384   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,386   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,388   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,390   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,391   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,393   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,395   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,396   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,398   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,400   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,401   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,404   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,406   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,408   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,410   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,412   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,414   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,416   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,418   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,420   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,423   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,425   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,427   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,429   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,431   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,433   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,435   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,437   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,439   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,441   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,443   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,445   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,446   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,448   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,450   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,452   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,454   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,456   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,458   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,460   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,461   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,463   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,465   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,467   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,469   copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot
2024-01-31T16:25:33,471   copying build/lib/lm_eval/tasks/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks
2024-01-31T16:25:33,473   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost
2024-01-31T16:25:33,474   copying build/lib/lm_eval/tasks/prost/corypaik_prost.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost
2024-01-31T16:25:33,476   copying build/lib/lm_eval/tasks/prost/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost
2024-01-31T16:25:33,478   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh
2024-01-31T16:25:33,480   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,481   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,482   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,484   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,486   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,487   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,489   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,491   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,492   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,494   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,496   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,498   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,500   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,501   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,504   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,505   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,507   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,509   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,511   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,513   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,515   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,517   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,518   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,521   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,522   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,524   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,526   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,527   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,529   copying build/lib/lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot
2024-01-31T16:25:33,531   copying build/lib/lm_eval/tasks/bbh/_generate_configs.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh
2024-01-31T16:25:33,534   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,534   copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,536   copying build/lib/lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,538   copying build/lib/lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,539   copying build/lib/lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,541   copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,543   copying build/lib/lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,545   copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,546   copying build/lib/lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,548   copying build/lib/lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,550   copying build/lib/lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,552   copying build/lib/lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,554   copying build/lib/lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,556   copying build/lib/lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,557   copying build/lib/lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,559   copying build/lib/lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,561   copying build/lib/lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,562   copying build/lib/lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,564   copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,566   copying build/lib/lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,568   copying build/lib/lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,569   copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,571   copying build/lib/lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,573   copying build/lib/lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,575   copying build/lib/lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,576   copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,578   copying build/lib/lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,580   copying build/lib/lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,581   copying build/lib/lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot
2024-01-31T16:25:33,583   copying build/lib/lm_eval/tasks/bbh/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh
2024-01-31T16:25:33,585   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,586   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,588   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,590   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,592   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,593   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,595   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,597   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,598   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,600   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,602   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,603   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,605   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,607   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,609   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,611   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,613   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,614   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,616   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,617   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,619   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,620   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,622   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,624   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,625   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,627   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,628   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,630   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,632   copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot
2024-01-31T16:25:33,634   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,635   copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,637   copying build/lib/lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,639   copying build/lib/lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,640   copying build/lib/lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,642   copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,644   copying build/lib/lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,645   copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,647   copying build/lib/lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,649   copying build/lib/lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,650   copying build/lib/lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,652   copying build/lib/lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,654   copying build/lib/lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,655   copying build/lib/lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,657   copying build/lib/lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,659   copying build/lib/lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,661   copying build/lib/lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,662   copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,664   copying build/lib/lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,665   copying build/lib/lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,667   copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,669   copying build/lib/lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,671   copying build/lib/lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,672   copying build/lib/lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,674   copying build/lib/lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,676   copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,677   copying build/lib/lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,679   copying build/lib/lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,681   copying build/lib/lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot
2024-01-31T16:25:33,683   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:33,684   copying build/lib/lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:33,686   copying build/lib/lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:33,687   copying build/lib/lm_eval/tasks/lambada_cloze/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze
2024-01-31T16:25:33,690   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue
2024-01-31T16:25:33,691   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:33,692   copying build/lib/lm_eval/tasks/super_glue/boolq/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:33,694   copying build/lib/lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:33,695   copying build/lib/lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq
2024-01-31T16:25:33,697   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:33,698   copying build/lib/lm_eval/tasks/super_glue/copa/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:33,700   copying build/lib/lm_eval/tasks/super_glue/copa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:33,702   copying build/lib/lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa
2024-01-31T16:25:33,704   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:33,705   copying build/lib/lm_eval/tasks/super_glue/multirc/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:33,707   copying build/lib/lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:33,709   copying build/lib/lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc
2024-01-31T16:25:33,711   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:33,711   copying build/lib/lm_eval/tasks/super_glue/cb/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:33,713   copying build/lib/lm_eval/tasks/super_glue/cb/aggregate.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:33,715   copying build/lib/lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:33,717   copying build/lib/lm_eval/tasks/super_glue/cb/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb
2024-01-31T16:25:33,719   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte
2024-01-31T16:25:33,720   copying build/lib/lm_eval/tasks/super_glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte
2024-01-31T16:25:33,721   copying build/lib/lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte
2024-01-31T16:25:33,723   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:33,724   copying build/lib/lm_eval/tasks/super_glue/wsc/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:33,726   copying build/lib/lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:33,728   copying build/lib/lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:33,729   copying build/lib/lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc
2024-01-31T16:25:33,732   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record
2024-01-31T16:25:33,733   copying build/lib/lm_eval/tasks/super_glue/record/util.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record
2024-01-31T16:25:33,735   copying build/lib/lm_eval/tasks/super_glue/record/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record
2024-01-31T16:25:33,736   copying build/lib/lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record
2024-01-31T16:25:33,738   copying build/lib/lm_eval/tasks/super_glue/record/t5_utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record
2024-01-31T16:25:33,740   copying build/lib/lm_eval/tasks/super_glue/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue
2024-01-31T16:25:33,742   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic
2024-01-31T16:25:33,743   copying build/lib/lm_eval/tasks/super_glue/wic/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic
2024-01-31T16:25:33,745   copying build/lib/lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic
2024-01-31T16:25:33,747   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa
2024-01-31T16:25:33,748   copying build/lib/lm_eval/tasks/openbookqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa
2024-01-31T16:25:33,750   copying build/lib/lm_eval/tasks/openbookqa/openbookqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa
2024-01-31T16:25:33,752   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre
2024-01-31T16:25:33,753   copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre
2024-01-31T16:25:33,754   copying build/lib/lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre
2024-01-31T16:25:33,756   copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre
2024-01-31T16:25:33,758   copying build/lib/lm_eval/tasks/qa4mre/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre
2024-01-31T16:25:33,759   copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre
2024-01-31T16:25:33,761   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,762   copying build/lib/lm_eval/tasks/xstorycloze/default_eu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,764   copying build/lib/lm_eval/tasks/xstorycloze/default_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,765   copying build/lib/lm_eval/tasks/xstorycloze/default_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,767   copying build/lib/lm_eval/tasks/xstorycloze/default_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,769   copying build/lib/lm_eval/tasks/xstorycloze/default_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,770   copying build/lib/lm_eval/tasks/xstorycloze/default_my.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,773   copying build/lib/lm_eval/tasks/xstorycloze/default_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,774   copying build/lib/lm_eval/tasks/xstorycloze/default_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,776   copying build/lib/lm_eval/tasks/xstorycloze/default_te.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,778   copying build/lib/lm_eval/tasks/xstorycloze/default_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,779   copying build/lib/lm_eval/tasks/xstorycloze/default_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,781   copying build/lib/lm_eval/tasks/xstorycloze/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze
2024-01-31T16:25:33,784   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,785   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,787   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,788   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,790   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,792   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,794   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,795   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,797   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_public_safety.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,799   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,800   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,802   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_information_technology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,804   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,805   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,807   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,808   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_real_estate.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,810   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_law.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,812   copying build/lib/lm_eval/tasks/kmmlu/_default_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,813   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_health.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,815   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_ecology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,817   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,818   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,820   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,822   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_food_processing.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,823   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,825   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,827   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_accounting.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,829   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,830   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,832   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_biology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,834   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_taxation.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,835   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_construction.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,837   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_economics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,839   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,840   copying build/lib/lm_eval/tasks/kmmlu/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,842   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_geomatics.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,844   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_energy_management.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,846   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_fashion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,847   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_patent.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,849   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,851   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,852   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,854   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_psychology.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,855   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,857   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_education.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,859   copying build/lib/lm_eval/tasks/kmmlu/kmmlu_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu
2024-01-31T16:25:33,861   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,862   copying build/lib/lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,863   copying build/lib/lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,865   copying build/lib/lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,867   copying build/lib/lm_eval/tasks/xwinograd/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,869   copying build/lib/lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,870   copying build/lib/lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,872   copying build/lib/lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,874   copying build/lib/lm_eval/tasks/xwinograd/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,876   copying build/lib/lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd
2024-01-31T16:25:33,878   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,879   copying build/lib/lm_eval/tasks/xnli/xnli_bg.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,881   copying build/lib/lm_eval/tasks/xnli/xnli_en.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,882   copying build/lib/lm_eval/tasks/xnli/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,885   copying build/lib/lm_eval/tasks/xnli/xnli_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,886   copying build/lib/lm_eval/tasks/xnli/xnli_fr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,888   copying build/lib/lm_eval/tasks/xnli/xnli_hi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,890   copying build/lib/lm_eval/tasks/xnli/xnli_tr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,892   copying build/lib/lm_eval/tasks/xnli/xnli_es.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,893   copying build/lib/lm_eval/tasks/xnli/xnli_ru.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,895   copying build/lib/lm_eval/tasks/xnli/xnli_ar.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,897   copying build/lib/lm_eval/tasks/xnli/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,899   copying build/lib/lm_eval/tasks/xnli/xnli_el.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,901   copying build/lib/lm_eval/tasks/xnli/xnli_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,902   copying build/lib/lm_eval/tasks/xnli/xnli_de.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,904   copying build/lib/lm_eval/tasks/xnli/xnli_ur.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,905   copying build/lib/lm_eval/tasks/xnli/xnli_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,907   copying build/lib/lm_eval/tasks/xnli/xnli_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,909   copying build/lib/lm_eval/tasks/xnli/xnli_common_yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli
2024-01-31T16:25:33,911   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble
2024-01-31T16:25:33,911   copying build/lib/lm_eval/tasks/unscramble/reversed_words.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble
2024-01-31T16:25:33,913   copying build/lib/lm_eval/tasks/unscramble/anagrams1.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble
2024-01-31T16:25:33,915   copying build/lib/lm_eval/tasks/unscramble/random_insertion.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble
2024-01-31T16:25:33,917   copying build/lib/lm_eval/tasks/unscramble/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble
2024-01-31T16:25:33,919   copying build/lib/lm_eval/tasks/unscramble/anagrams2.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble
2024-01-31T16:25:33,920   copying build/lib/lm_eval/tasks/unscramble/cycle_letters.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble
2024-01-31T16:25:33,922   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco
2024-01-31T16:25:33,923   copying build/lib/lm_eval/tasks/mc_taco/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco
2024-01-31T16:25:33,925   copying build/lib/lm_eval/tasks/mc_taco/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco
2024-01-31T16:25:33,928   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,929   copying build/lib/lm_eval/tasks/xcopa/default_sw.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,930   copying build/lib/lm_eval/tasks/xcopa/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,932   copying build/lib/lm_eval/tasks/xcopa/default_th.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,934   copying build/lib/lm_eval/tasks/xcopa/default_vi.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,936   copying build/lib/lm_eval/tasks/xcopa/default_ht.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,937   copying build/lib/lm_eval/tasks/xcopa/default_id.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,939   copying build/lib/lm_eval/tasks/xcopa/default_zh.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,941   copying build/lib/lm_eval/tasks/xcopa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,942   copying build/lib/lm_eval/tasks/xcopa/default_tr.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,944   copying build/lib/lm_eval/tasks/xcopa/default_et.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,946   copying build/lib/lm_eval/tasks/xcopa/default_qu.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,948   copying build/lib/lm_eval/tasks/xcopa/default_ta.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,950   copying build/lib/lm_eval/tasks/xcopa/default_it.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa
2024-01-31T16:25:33,952   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273
2024-01-31T16:25:33,953   copying build/lib/lm_eval/tasks/wsc273/default.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273
2024-01-31T16:25:33,955   copying build/lib/lm_eval/tasks/wsc273/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273
2024-01-31T16:25:33,956   copying build/lib/lm_eval/tasks/wsc273/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273
2024-01-31T16:25:33,958   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/race
2024-01-31T16:25:33,959   copying build/lib/lm_eval/tasks/race/race.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/race
2024-01-31T16:25:33,961   copying build/lib/lm_eval/tasks/race/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/race
2024-01-31T16:25:33,963   copying build/lib/lm_eval/tasks/race/preprocess_race.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/race
2024-01-31T16:25:33,964   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa
2024-01-31T16:25:33,965   copying build/lib/lm_eval/tasks/logiqa/utils_logiqa.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa
2024-01-31T16:25:33,967   copying build/lib/lm_eval/tasks/logiqa/logiqa.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa
2024-01-31T16:25:33,969   copying build/lib/lm_eval/tasks/logiqa/README.md -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa
2024-01-31T16:25:33,971   creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts
2024-01-31T16:25:33,972   copying build/lib/lm_eval/tasks/realtoxicityprompts/metric.py -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts
2024-01-31T16:25:33,973   copying build/lib/lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts
2024-01-31T16:25:33,975   copying build/lib/lm_eval/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval
2024-01-31T16:25:33,977   copying build/lib/lm_eval/utils.py -> build/bdist.linux-armv7l/wheel/lm_eval
2024-01-31T16:25:33,980   creating build/bdist.linux-armv7l/wheel/lm_eval/prompts
2024-01-31T16:25:33,981   copying build/lib/lm_eval/prompts/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/prompts
2024-01-31T16:25:33,983   creating build/bdist.linux-armv7l/wheel/lm_eval/decontamination
2024-01-31T16:25:33,984   copying build/lib/lm_eval/decontamination/decontaminate.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination
2024-01-31T16:25:33,986   copying build/lib/lm_eval/decontamination/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination
2024-01-31T16:25:33,988   copying build/lib/lm_eval/decontamination/janitor.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination
2024-01-31T16:25:33,990   copying build/lib/lm_eval/decontamination/archiver.py -> build/bdist.linux-armv7l/wheel/lm_eval/decontamination
2024-01-31T16:25:33,992   creating build/bdist.linux-armv7l/wheel/lm_eval/filters
2024-01-31T16:25:33,993   copying build/lib/lm_eval/filters/decontamination.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters
2024-01-31T16:25:33,995   copying build/lib/lm_eval/filters/extraction.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters
2024-01-31T16:25:33,997   copying build/lib/lm_eval/filters/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters
2024-01-31T16:25:33,999   copying build/lib/lm_eval/filters/transformation.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters
2024-01-31T16:25:34,000   copying build/lib/lm_eval/filters/selection.py -> build/bdist.linux-armv7l/wheel/lm_eval/filters
2024-01-31T16:25:34,002   copying build/lib/lm_eval/__main__.py -> build/bdist.linux-armv7l/wheel/lm_eval
2024-01-31T16:25:34,005   creating build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,006   copying build/lib/lm_eval/models/huggingface.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,009   copying build/lib/lm_eval/models/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,011   copying build/lib/lm_eval/models/optimum_lm.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,013   copying build/lib/lm_eval/models/mamba_lm.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,015   copying build/lib/lm_eval/models/gguf.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,017   copying build/lib/lm_eval/models/anthropic_llms.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,018   copying build/lib/lm_eval/models/openai_completions.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,021   copying build/lib/lm_eval/models/vllm_causallms.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,023   copying build/lib/lm_eval/models/textsynth.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,025   copying build/lib/lm_eval/models/dummy.py -> build/bdist.linux-armv7l/wheel/lm_eval/models
2024-01-31T16:25:34,027   copying build/lib/lm_eval/evaluator.py -> build/bdist.linux-armv7l/wheel/lm_eval
2024-01-31T16:25:34,030   creating build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,031   copying build/lib/lm_eval/api/model.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,033   copying build/lib/lm_eval/api/metrics.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,035   copying build/lib/lm_eval/api/__init__.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,036   copying build/lib/lm_eval/api/samplers.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,038   copying build/lib/lm_eval/api/instance.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,040   copying build/lib/lm_eval/api/task.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,042   copying build/lib/lm_eval/api/registry.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,044   copying build/lib/lm_eval/api/filter.py -> build/bdist.linux-armv7l/wheel/lm_eval/api
2024-01-31T16:25:34,046   running install_egg_info
2024-01-31T16:25:34,050   Copying lm_eval.egg-info to build/bdist.linux-armv7l/wheel/lm_eval-0.4.1-py3.11.egg-info
2024-01-31T16:25:34,063   running install_scripts
2024-01-31T16:25:34,096   creating build/bdist.linux-armv7l/wheel/lm_eval-0.4.1.dist-info/WHEEL
2024-01-31T16:25:34,098   creating '/tmp/pip-wheel-sd4il15g/.tmp-nw9pub27/lm_eval-0.4.1-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it
2024-01-31T16:25:34,100   adding 'lm_eval/__init__.py'
2024-01-31T16:25:34,102   adding 'lm_eval/__main__.py'
2024-01-31T16:25:34,105   adding 'lm_eval/evaluator.py'
2024-01-31T16:25:34,110   adding 'lm_eval/utils.py'
2024-01-31T16:25:34,112   adding 'lm_eval/api/__init__.py'
2024-01-31T16:25:34,113   adding 'lm_eval/api/filter.py'
2024-01-31T16:25:34,114   adding 'lm_eval/api/instance.py'
2024-01-31T16:25:34,116   adding 'lm_eval/api/metrics.py'
2024-01-31T16:25:34,118   adding 'lm_eval/api/model.py'
2024-01-31T16:25:34,120   adding 'lm_eval/api/registry.py'
2024-01-31T16:25:34,121   adding 'lm_eval/api/samplers.py'
2024-01-31T16:25:34,127   adding 'lm_eval/api/task.py'
2024-01-31T16:25:34,129   adding 'lm_eval/decontamination/__init__.py'
2024-01-31T16:25:34,130   adding 'lm_eval/decontamination/archiver.py'
2024-01-31T16:25:34,132   adding 'lm_eval/decontamination/decontaminate.py'
2024-01-31T16:25:34,134   adding 'lm_eval/decontamination/janitor.py'
2024-01-31T16:25:34,136   adding 'lm_eval/filters/__init__.py'
2024-01-31T16:25:34,137   adding 'lm_eval/filters/decontamination.py'
2024-01-31T16:25:34,139   adding 'lm_eval/filters/extraction.py'
2024-01-31T16:25:34,140   adding 'lm_eval/filters/selection.py'
2024-01-31T16:25:34,141   adding 'lm_eval/filters/transformation.py'
2024-01-31T16:25:34,143   adding 'lm_eval/models/__init__.py'
2024-01-31T16:25:34,145   adding 'lm_eval/models/anthropic_llms.py'
2024-01-31T16:25:34,146   adding 'lm_eval/models/dummy.py'
2024-01-31T16:25:34,147   adding 'lm_eval/models/gguf.py'
2024-01-31T16:25:34,153   adding 'lm_eval/models/huggingface.py'
2024-01-31T16:25:34,155   adding 'lm_eval/models/mamba_lm.py'
2024-01-31T16:25:34,157   adding 'lm_eval/models/openai_completions.py'
2024-01-31T16:25:34,158   adding 'lm_eval/models/optimum_lm.py'
2024-01-31T16:25:34,160   adding 'lm_eval/models/textsynth.py'
2024-01-31T16:25:34,162   adding 'lm_eval/models/vllm_causallms.py'
2024-01-31T16:25:34,164   adding 'lm_eval/prompts/__init__.py'
2024-01-31T16:25:34,167   adding 'lm_eval/tasks/README.md'
2024-01-31T16:25:34,169   adding 'lm_eval/tasks/__init__.py'
2024-01-31T16:25:34,171   adding 'lm_eval/tasks/anli/README.md'
2024-01-31T16:25:34,172   adding 'lm_eval/tasks/anli/anli_r1.yaml'
2024-01-31T16:25:34,173   adding 'lm_eval/tasks/anli/anli_r2.yaml'
2024-01-31T16:25:34,174   adding 'lm_eval/tasks/anli/anli_r3.yaml'
2024-01-31T16:25:34,176   adding 'lm_eval/tasks/arc/README.md'
2024-01-31T16:25:34,177   adding 'lm_eval/tasks/arc/arc_challenge.yaml'
2024-01-31T16:25:34,178   adding 'lm_eval/tasks/arc/arc_easy.yaml'
2024-01-31T16:25:34,180   adding 'lm_eval/tasks/arithmetic/README.md'
2024-01-31T16:25:34,181   adding 'lm_eval/tasks/arithmetic/arithmetic_1dc.yaml'
2024-01-31T16:25:34,182   adding 'lm_eval/tasks/arithmetic/arithmetic_2da.yaml'
2024-01-31T16:25:34,183   adding 'lm_eval/tasks/arithmetic/arithmetic_2dm.yaml'
2024-01-31T16:25:34,184   adding 'lm_eval/tasks/arithmetic/arithmetic_2ds.yaml'
2024-01-31T16:25:34,185   adding 'lm_eval/tasks/arithmetic/arithmetic_3da.yaml'
2024-01-31T16:25:34,186   adding 'lm_eval/tasks/arithmetic/arithmetic_3ds.yaml'
2024-01-31T16:25:34,187   adding 'lm_eval/tasks/arithmetic/arithmetic_4da.yaml'
2024-01-31T16:25:34,189   adding 'lm_eval/tasks/arithmetic/arithmetic_4ds.yaml'
2024-01-31T16:25:34,190   adding 'lm_eval/tasks/arithmetic/arithmetic_5da.yaml'
2024-01-31T16:25:34,191   adding 'lm_eval/tasks/arithmetic/arithmetic_5ds.yaml'
2024-01-31T16:25:34,193   adding 'lm_eval/tasks/asdiv/README.md'
2024-01-31T16:25:34,194   adding 'lm_eval/tasks/asdiv/default.yaml'
2024-01-31T16:25:34,196   adding 'lm_eval/tasks/babi/README.md'
2024-01-31T16:25:34,197   adding 'lm_eval/tasks/babi/babi.yaml'
2024-01-31T16:25:34,199   adding 'lm_eval/tasks/bbh/README.md'
2024-01-31T16:25:34,200   adding 'lm_eval/tasks/bbh/_generate_configs.py'
2024-01-31T16:25:34,202   adding 'lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml'
2024-01-31T16:25:34,203   adding 'lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml'
2024-01-31T16:25:34,205   adding 'lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml'
2024-01-31T16:25:34,206   adding 'lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml'
2024-01-31T16:25:34,207   adding 'lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml'
2024-01-31T16:25:34,209   adding 'lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml'
2024-01-31T16:25:34,210   adding 'lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml'
2024-01-31T16:25:34,212   adding 'lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml'
2024-01-31T16:25:34,213   adding 'lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml'
2024-01-31T16:25:34,214   adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml'
2024-01-31T16:25:34,216   adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml'
2024-01-31T16:25:34,217   adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml'
2024-01-31T16:25:34,219   adding 'lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml'
2024-01-31T16:25:34,220   adding 'lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml'
2024-01-31T16:25:34,221   adding 'lm_eval/tasks/bbh/cot_fewshot/navigate.yaml'
2024-01-31T16:25:34,223   adding 'lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml'
2024-01-31T16:25:34,224   adding 'lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml'
2024-01-31T16:25:34,225   adding 'lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml'
2024-01-31T16:25:34,227   adding 'lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml'
2024-01-31T16:25:34,228   adding 'lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml'
2024-01-31T16:25:34,229   adding 'lm_eval/tasks/bbh/cot_fewshot/snarks.yaml'
2024-01-31T16:25:34,231   adding 'lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml'
2024-01-31T16:25:34,232   adding 'lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml'
2024-01-31T16:25:34,233   adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml'
2024-01-31T16:25:34,234   adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml'
2024-01-31T16:25:34,235   adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml'
2024-01-31T16:25:34,237   adding 'lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml'
2024-01-31T16:25:34,238   adding 'lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml'
2024-01-31T16:25:34,240   adding 'lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml'
2024-01-31T16:25:34,241   adding 'lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml'
2024-01-31T16:25:34,242   adding 'lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml'
2024-01-31T16:25:34,243   adding 'lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml'
2024-01-31T16:25:34,244   adding 'lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml'
2024-01-31T16:25:34,245   adding 'lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml'
2024-01-31T16:25:34,246   adding 'lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml'
2024-01-31T16:25:34,247   adding 'lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml'
2024-01-31T16:25:34,248   adding 'lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml'
2024-01-31T16:25:34,250   adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml'
2024-01-31T16:25:34,251   adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml'
2024-01-31T16:25:34,252   adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml'
2024-01-31T16:25:34,253   adding 'lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml'
2024-01-31T16:25:34,254   adding 'lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml'
2024-01-31T16:25:34,255   adding 'lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml'
2024-01-31T16:25:34,256   adding 'lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml'
2024-01-31T16:25:34,258   adding 'lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml'
2024-01-31T16:25:34,259   adding 'lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml'
2024-01-31T16:25:34,260   adding 'lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml'
2024-01-31T16:25:34,261   adding 'lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml'
2024-01-31T16:25:34,262   adding 'lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml'
2024-01-31T16:25:34,263   adding 'lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml'
2024-01-31T16:25:34,264   adding 'lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml'
2024-01-31T16:25:34,265   adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml'
2024-01-31T16:25:34,267   adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml'
2024-01-31T16:25:34,268   adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml'
2024-01-31T16:25:34,269   adding 'lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml'
2024-01-31T16:25:34,270   adding 'lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml'
2024-01-31T16:25:34,272   adding 'lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml'
2024-01-31T16:25:34,273   adding 'lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml'
2024-01-31T16:25:34,275   adding 'lm_eval/tasks/bbh/fewshot/causal_judgement.yaml'
2024-01-31T16:25:34,276   adding 'lm_eval/tasks/bbh/fewshot/date_understanding.yaml'
2024-01-31T16:25:34,277   adding 'lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml'
2024-01-31T16:25:34,278   adding 'lm_eval/tasks/bbh/fewshot/dyck_languages.yaml'
2024-01-31T16:25:34,279   adding 'lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml'
2024-01-31T16:25:34,280   adding 'lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml'
2024-01-31T16:25:34,282   adding 'lm_eval/tasks/bbh/fewshot/hyperbaton.yaml'
2024-01-31T16:25:34,283   adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml'
2024-01-31T16:25:34,284   adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml'
2024-01-31T16:25:34,285   adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml'
2024-01-31T16:25:34,286   adding 'lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml'
2024-01-31T16:25:34,287   adding 'lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml'
2024-01-31T16:25:34,289   adding 'lm_eval/tasks/bbh/fewshot/navigate.yaml'
2024-01-31T16:25:34,290   adding 'lm_eval/tasks/bbh/fewshot/object_counting.yaml'
2024-01-31T16:25:34,291   adding 'lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml'
2024-01-31T16:25:34,292   adding 'lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml'
2024-01-31T16:25:34,293   adding 'lm_eval/tasks/bbh/fewshot/ruin_names.yaml'
2024-01-31T16:25:34,294   adding 'lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml'
2024-01-31T16:25:34,295   adding 'lm_eval/tasks/bbh/fewshot/snarks.yaml'
2024-01-31T16:25:34,296   adding 'lm_eval/tasks/bbh/fewshot/sports_understanding.yaml'
2024-01-31T16:25:34,298   adding 'lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml'
2024-01-31T16:25:34,299   adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml'
2024-01-31T16:25:34,300   adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml'
2024-01-31T16:25:34,302   adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml'
2024-01-31T16:25:34,303   adding 'lm_eval/tasks/bbh/fewshot/web_of_lies.yaml'
2024-01-31T16:25:34,304   adding 'lm_eval/tasks/bbh/fewshot/word_sorting.yaml'
2024-01-31T16:25:34,306   adding 'lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml'
2024-01-31T16:25:34,307   adding 'lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml'
2024-01-31T16:25:34,308   adding 'lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml'
2024-01-31T16:25:34,309   adding 'lm_eval/tasks/bbh/zeroshot/date_understanding.yaml'
2024-01-31T16:25:34,310   adding 'lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml'
2024-01-31T16:25:34,311   adding 'lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml'
2024-01-31T16:25:34,313   adding 'lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml'
2024-01-31T16:25:34,314   adding 'lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml'
2024-01-31T16:25:34,315   adding 'lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml'
2024-01-31T16:25:34,316   adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml'
2024-01-31T16:25:34,317   adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml'
2024-01-31T16:25:34,318   adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml'
2024-01-31T16:25:34,319   adding 'lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml'
2024-01-31T16:25:34,321   adding 'lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml'
2024-01-31T16:25:34,322   adding 'lm_eval/tasks/bbh/zeroshot/navigate.yaml'
2024-01-31T16:25:34,323   adding 'lm_eval/tasks/bbh/zeroshot/object_counting.yaml'
2024-01-31T16:25:34,324   adding 'lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml'
2024-01-31T16:25:34,325   adding 'lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml'
2024-01-31T16:25:34,326   adding 'lm_eval/tasks/bbh/zeroshot/ruin_names.yaml'
2024-01-31T16:25:34,327   adding 'lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml'
2024-01-31T16:25:34,329   adding 'lm_eval/tasks/bbh/zeroshot/snarks.yaml'
2024-01-31T16:25:34,330   adding 'lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml'
2024-01-31T16:25:34,331   adding 'lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml'
2024-01-31T16:25:34,332   adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml'
2024-01-31T16:25:34,334   adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml'
2024-01-31T16:25:34,335   adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml'
2024-01-31T16:25:34,337   adding 'lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml'
2024-01-31T16:25:34,338   adding 'lm_eval/tasks/bbh/zeroshot/word_sorting.yaml'
2024-01-31T16:25:34,341   adding 'lm_eval/tasks/belebele/README.md'
2024-01-31T16:25:34,343   adding 'lm_eval/tasks/belebele/_default_template_yaml'
2024-01-31T16:25:34,344   adding 'lm_eval/tasks/belebele/_generate_configs.py'
2024-01-31T16:25:34,345   adding 'lm_eval/tasks/belebele/belebele_acm_Arab.yaml'
2024-01-31T16:25:34,346   adding 'lm_eval/tasks/belebele/belebele_afr_Latn.yaml'
2024-01-31T16:25:34,347   adding 'lm_eval/tasks/belebele/belebele_als_Latn.yaml'
2024-01-31T16:25:34,348   adding 'lm_eval/tasks/belebele/belebele_amh_Ethi.yaml'
2024-01-31T16:25:34,349   adding 'lm_eval/tasks/belebele/belebele_apc_Arab.yaml'
2024-01-31T16:25:34,350   adding 'lm_eval/tasks/belebele/belebele_arb_Arab.yaml'
2024-01-31T16:25:34,351   adding 'lm_eval/tasks/belebele/belebele_arb_Latn.yaml'
2024-01-31T16:25:34,352   adding 'lm_eval/tasks/belebele/belebele_ars_Arab.yaml'
2024-01-31T16:25:34,353   adding 'lm_eval/tasks/belebele/belebele_ary_Arab.yaml'
2024-01-31T16:25:34,354   adding 'lm_eval/tasks/belebele/belebele_arz_Arab.yaml'
2024-01-31T16:25:34,355   adding 'lm_eval/tasks/belebele/belebele_asm_Beng.yaml'
2024-01-31T16:25:34,356   adding 'lm_eval/tasks/belebele/belebele_azj_Latn.yaml'
2024-01-31T16:25:34,357   adding 'lm_eval/tasks/belebele/belebele_bam_Latn.yaml'
2024-01-31T16:25:34,358   adding 'lm_eval/tasks/belebele/belebele_ben_Beng.yaml'
2024-01-31T16:25:34,359   adding 'lm_eval/tasks/belebele/belebele_ben_Latn.yaml'
2024-01-31T16:25:34,360   adding 'lm_eval/tasks/belebele/belebele_bod_Tibt.yaml'
2024-01-31T16:25:34,361   adding 'lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml'
2024-01-31T16:25:34,362   adding 'lm_eval/tasks/belebele/belebele_cat_Latn.yaml'
2024-01-31T16:25:34,363   adding 'lm_eval/tasks/belebele/belebele_ceb_Latn.yaml'
2024-01-31T16:25:34,364   adding 'lm_eval/tasks/belebele/belebele_ces_Latn.yaml'
2024-01-31T16:25:34,365   adding 'lm_eval/tasks/belebele/belebele_ckb_Arab.yaml'
2024-01-31T16:25:34,366   adding 'lm_eval/tasks/belebele/belebele_dan_Latn.yaml'
2024-01-31T16:25:34,367   adding 'lm_eval/tasks/belebele/belebele_default.yaml'
2024-01-31T16:25:34,369   adding 'lm_eval/tasks/belebele/belebele_deu_Latn.yaml'
2024-01-31T16:25:34,370   adding 'lm_eval/tasks/belebele/belebele_ell_Grek.yaml'
2024-01-31T16:25:34,371   adding 'lm_eval/tasks/belebele/belebele_eng_Latn.yaml'
2024-01-31T16:25:34,372   adding 'lm_eval/tasks/belebele/belebele_est_Latn.yaml'
2024-01-31T16:25:34,373   adding 'lm_eval/tasks/belebele/belebele_eus_Latn.yaml'
2024-01-31T16:25:34,374   adding 'lm_eval/tasks/belebele/belebele_fin_Latn.yaml'
2024-01-31T16:25:34,375   adding 'lm_eval/tasks/belebele/belebele_fra_Latn.yaml'
2024-01-31T16:25:34,376   adding 'lm_eval/tasks/belebele/belebele_fuv_Latn.yaml'
2024-01-31T16:25:34,377   adding 'lm_eval/tasks/belebele/belebele_gaz_Latn.yaml'
2024-01-31T16:25:34,378   adding 'lm_eval/tasks/belebele/belebele_grn_Latn.yaml'
2024-01-31T16:25:34,380   adding 'lm_eval/tasks/belebele/belebele_guj_Gujr.yaml'
2024-01-31T16:25:34,381   adding 'lm_eval/tasks/belebele/belebele_hat_Latn.yaml'
2024-01-31T16:25:34,382   adding 'lm_eval/tasks/belebele/belebele_hau_Latn.yaml'
2024-01-31T16:25:34,383   adding 'lm_eval/tasks/belebele/belebele_heb_Hebr.yaml'
2024-01-31T16:25:34,384   adding 'lm_eval/tasks/belebele/belebele_hin_Deva.yaml'
2024-01-31T16:25:34,385   adding 'lm_eval/tasks/belebele/belebele_hin_Latn.yaml'
2024-01-31T16:25:34,386   adding 'lm_eval/tasks/belebele/belebele_hrv_Latn.yaml'
2024-01-31T16:25:34,387   adding 'lm_eval/tasks/belebele/belebele_hun_Latn.yaml'
2024-01-31T16:25:34,388   adding 'lm_eval/tasks/belebele/belebele_hye_Armn.yaml'
2024-01-31T16:25:34,390   adding 'lm_eval/tasks/belebele/belebele_ibo_Latn.yaml'
2024-01-31T16:25:34,391   adding 'lm_eval/tasks/belebele/belebele_ilo_Latn.yaml'
2024-01-31T16:25:34,392   adding 'lm_eval/tasks/belebele/belebele_ind_Latn.yaml'
2024-01-31T16:25:34,393   adding 'lm_eval/tasks/belebele/belebele_isl_Latn.yaml'
2024-01-31T16:25:34,394   adding 'lm_eval/tasks/belebele/belebele_ita_Latn.yaml'
2024-01-31T16:25:34,395   adding 'lm_eval/tasks/belebele/belebele_jav_Latn.yaml'
2024-01-31T16:25:34,396   adding 'lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml'
2024-01-31T16:25:34,397   adding 'lm_eval/tasks/belebele/belebele_kac_Latn.yaml'
2024-01-31T16:25:34,398   adding 'lm_eval/tasks/belebele/belebele_kan_Knda.yaml'
2024-01-31T16:25:34,399   adding 'lm_eval/tasks/belebele/belebele_kat_Geor.yaml'
2024-01-31T16:25:34,400   adding 'lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml'
2024-01-31T16:25:34,401   adding 'lm_eval/tasks/belebele/belebele_kea_Latn.yaml'
2024-01-31T16:25:34,402   adding 'lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml'
2024-01-31T16:25:34,403   adding 'lm_eval/tasks/belebele/belebele_khm_Khmr.yaml'
2024-01-31T16:25:34,404   adding 'lm_eval/tasks/belebele/belebele_kin_Latn.yaml'
2024-01-31T16:25:34,405   adding 'lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml'
2024-01-31T16:25:34,406   adding 'lm_eval/tasks/belebele/belebele_kor_Hang.yaml'
2024-01-31T16:25:34,407   adding 'lm_eval/tasks/belebele/belebele_lao_Laoo.yaml'
2024-01-31T16:25:34,409   adding 'lm_eval/tasks/belebele/belebele_lin_Latn.yaml'
2024-01-31T16:25:34,410   adding 'lm_eval/tasks/belebele/belebele_lit_Latn.yaml'
2024-01-31T16:25:34,411   adding 'lm_eval/tasks/belebele/belebele_lug_Latn.yaml'
2024-01-31T16:25:34,412   adding 'lm_eval/tasks/belebele/belebele_luo_Latn.yaml'
2024-01-31T16:25:34,413   adding 'lm_eval/tasks/belebele/belebele_lvs_Latn.yaml'
2024-01-31T16:25:34,414   adding 'lm_eval/tasks/belebele/belebele_mal_Mlym.yaml'
2024-01-31T16:25:34,415   adding 'lm_eval/tasks/belebele/belebele_mar_Deva.yaml'
2024-01-31T16:25:34,416   adding 'lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml'
2024-01-31T16:25:34,417   adding 'lm_eval/tasks/belebele/belebele_mlt_Latn.yaml'
2024-01-31T16:25:34,418   adding 'lm_eval/tasks/belebele/belebele_mri_Latn.yaml'
2024-01-31T16:25:34,419   adding 'lm_eval/tasks/belebele/belebele_mya_Mymr.yaml'
2024-01-31T16:25:34,420   adding 'lm_eval/tasks/belebele/belebele_nld_Latn.yaml'
2024-01-31T16:25:34,421   adding 'lm_eval/tasks/belebele/belebele_nob_Latn.yaml'
2024-01-31T16:25:34,422   adding 'lm_eval/tasks/belebele/belebele_npi_Deva.yaml'
2024-01-31T16:25:34,423   adding 'lm_eval/tasks/belebele/belebele_npi_Latn.yaml'
2024-01-31T16:25:34,425   adding 'lm_eval/tasks/belebele/belebele_nso_Latn.yaml'
2024-01-31T16:25:34,426   adding 'lm_eval/tasks/belebele/belebele_nya_Latn.yaml'
2024-01-31T16:25:34,427   adding 'lm_eval/tasks/belebele/belebele_ory_Orya.yaml'
2024-01-31T16:25:34,428   adding 'lm_eval/tasks/belebele/belebele_pan_Guru.yaml'
2024-01-31T16:25:34,429   adding 'lm_eval/tasks/belebele/belebele_pbt_Arab.yaml'
2024-01-31T16:25:34,430   adding 'lm_eval/tasks/belebele/belebele_pes_Arab.yaml'
2024-01-31T16:25:34,431   adding 'lm_eval/tasks/belebele/belebele_plt_Latn.yaml'
2024-01-31T16:25:34,432   adding 'lm_eval/tasks/belebele/belebele_pol_Latn.yaml'
2024-01-31T16:25:34,433   adding 'lm_eval/tasks/belebele/belebele_por_Latn.yaml'
2024-01-31T16:25:34,435   adding 'lm_eval/tasks/belebele/belebele_ron_Latn.yaml'
2024-01-31T16:25:34,436   adding 'lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml'
2024-01-31T16:25:34,437   adding 'lm_eval/tasks/belebele/belebele_shn_Mymr.yaml'
2024-01-31T16:25:34,438   adding 'lm_eval/tasks/belebele/belebele_sin_Latn.yaml'
2024-01-31T16:25:34,439   adding 'lm_eval/tasks/belebele/belebele_sin_Sinh.yaml'
2024-01-31T16:25:34,440   adding 'lm_eval/tasks/belebele/belebele_slk_Latn.yaml'
2024-01-31T16:25:34,441   adding 'lm_eval/tasks/belebele/belebele_slv_Latn.yaml'
2024-01-31T16:25:34,442   adding 'lm_eval/tasks/belebele/belebele_sna_Latn.yaml'
2024-01-31T16:25:34,443   adding 'lm_eval/tasks/belebele/belebele_snd_Arab.yaml'
2024-01-31T16:25:34,445   adding 'lm_eval/tasks/belebele/belebele_som_Latn.yaml'
2024-01-31T16:25:34,446   adding 'lm_eval/tasks/belebele/belebele_sot_Latn.yaml'
2024-01-31T16:25:34,447   adding 'lm_eval/tasks/belebele/belebele_spa_Latn.yaml'
2024-01-31T16:25:34,448   adding 'lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml'
2024-01-31T16:25:34,449   adding 'lm_eval/tasks/belebele/belebele_ssw_Latn.yaml'
2024-01-31T16:25:34,450   adding 'lm_eval/tasks/belebele/belebele_sun_Latn.yaml'
2024-01-31T16:25:34,451   adding 'lm_eval/tasks/belebele/belebele_swe_Latn.yaml'
2024-01-31T16:25:34,452   adding 'lm_eval/tasks/belebele/belebele_swh_Latn.yaml'
2024-01-31T16:25:34,453   adding 'lm_eval/tasks/belebele/belebele_tam_Taml.yaml'
2024-01-31T16:25:34,455   adding 'lm_eval/tasks/belebele/belebele_tel_Telu.yaml'
2024-01-31T16:25:34,456   adding 'lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml'
2024-01-31T16:25:34,457   adding 'lm_eval/tasks/belebele/belebele_tgl_Latn.yaml'
2024-01-31T16:25:34,458   adding 'lm_eval/tasks/belebele/belebele_tha_Thai.yaml'
2024-01-31T16:25:34,459   adding 'lm_eval/tasks/belebele/belebele_tir_Ethi.yaml'
2024-01-31T16:25:34,460   adding 'lm_eval/tasks/belebele/belebele_tsn_Latn.yaml'
2024-01-31T16:25:34,461   adding 'lm_eval/tasks/belebele/belebele_tso_Latn.yaml'
2024-01-31T16:25:34,462   adding 'lm_eval/tasks/belebele/belebele_tur_Latn.yaml'
2024-01-31T16:25:34,463   adding 'lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml'
2024-01-31T16:25:34,464   adding 'lm_eval/tasks/belebele/belebele_urd_Arab.yaml'
2024-01-31T16:25:34,465   adding 'lm_eval/tasks/belebele/belebele_urd_Latn.yaml'
2024-01-31T16:25:34,467   adding 'lm_eval/tasks/belebele/belebele_uzn_Latn.yaml'
2024-01-31T16:25:34,468   adding 'lm_eval/tasks/belebele/belebele_vie_Latn.yaml'
2024-01-31T16:25:34,469   adding 'lm_eval/tasks/belebele/belebele_war_Latn.yaml'
2024-01-31T16:25:34,470   adding 'lm_eval/tasks/belebele/belebele_wol_Latn.yaml'
2024-01-31T16:25:34,471   adding 'lm_eval/tasks/belebele/belebele_xho_Latn.yaml'
2024-01-31T16:25:34,472   adding 'lm_eval/tasks/belebele/belebele_yor_Latn.yaml'
2024-01-31T16:25:34,473   adding 'lm_eval/tasks/belebele/belebele_zho_Hans.yaml'
2024-01-31T16:25:34,474   adding 'lm_eval/tasks/belebele/belebele_zho_Hant.yaml'
2024-01-31T16:25:34,475   adding 'lm_eval/tasks/belebele/belebele_zsm_Latn.yaml'
2024-01-31T16:25:34,477   adding 'lm_eval/tasks/belebele/belebele_zul_Latn.yaml'
2024-01-31T16:25:34,478   adding 'lm_eval/tasks/benchmarks/minerva_math.yaml'
2024-01-31T16:25:34,479   adding 'lm_eval/tasks/benchmarks/pythia.yaml'
2024-01-31T16:25:34,481   adding 'lm_eval/tasks/benchmarks/t0_eval.yaml'
2024-01-31T16:25:34,482   adding 'lm_eval/tasks/benchmarks/flan/flan_anli.yaml'
2024-01-31T16:25:34,483   adding 'lm_eval/tasks/benchmarks/flan/flan_arc.yaml'
2024-01-31T16:25:34,485   adding 'lm_eval/tasks/benchmarks/flan/flan_boolq.yaml'
2024-01-31T16:25:34,486   adding 'lm_eval/tasks/benchmarks/flan/flan_cot.yaml'
2024-01-31T16:25:34,487   adding 'lm_eval/tasks/benchmarks/flan/flan_held_in.yaml'
2024-01-31T16:25:34,488   adding 'lm_eval/tasks/benchmarks/flan/flan_held_in_yaml'
2024-01-31T16:25:34,489   adding 'lm_eval/tasks/benchmarks/flan/flan_held_out.yaml'
2024-01-31T16:25:34,490   adding 'lm_eval/tasks/benchmarks/flan/flan_rte.yaml'
2024-01-31T16:25:34,492   adding 'lm_eval/tasks/benchmarks/flan/prompt_templates/anli.yaml'
2024-01-31T16:25:34,493   adding 'lm_eval/tasks/benchmarks/flan/prompt_templates/arc.yaml'
2024-01-31T16:25:34,494   adding 'lm_eval/tasks/benchmarks/flan/prompt_templates/boolq.yaml'
2024-01-31T16:25:34,496   adding 'lm_eval/tasks/benchmarks/flan/prompt_templates/rte.yaml'
2024-01-31T16:25:34,497   adding 'lm_eval/tasks/benchmarks/flan/yaml_templates/cot_template_yaml'
2024-01-31T16:25:34,499   adding 'lm_eval/tasks/benchmarks/flan/yaml_templates/held_in_template_yaml'
2024-01-31T16:25:34,501   adding 'lm_eval/tasks/benchmarks/multimedqa/README.md'
2024-01-31T16:25:34,502   adding 'lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml'
2024-01-31T16:25:34,505   adding 'lm_eval/tasks/bigbench/README.md'
2024-01-31T16:25:34,506   adding 'lm_eval/tasks/bigbench/generate_tasks.py'
2024-01-31T16:25:34,507   adding 'lm_eval/tasks/bigbench/generate_until_template_yaml'
2024-01-31T16:25:34,509   adding 'lm_eval/tasks/bigbench/multiple_choice_template_yaml'
2024-01-31T16:25:34,510   adding 'lm_eval/tasks/bigbench/push_bigbench_dataset.py'
2024-01-31T16:25:34,514   adding 'lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml'
2024-01-31T16:25:34,515   adding 'lm_eval/tasks/bigbench/generate_until/anachronisms.yaml'
2024-01-31T16:25:34,517   adding 'lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml'
2024-01-31T16:25:34,518   adding 'lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml'
2024-01-31T16:25:34,518   adding 'lm_eval/tasks/bigbench/generate_until/arithmetic.yaml'
2024-01-31T16:25:34,519   adding 'lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml'
2024-01-31T16:25:34,520   adding 'lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml'
2024-01-31T16:25:34,521   adding 'lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml'
2024-01-31T16:25:34,522   adding 'lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml'
2024-01-31T16:25:34,523   adding 'lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml'
2024-01-31T16:25:34,525   adding 'lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml'
2024-01-31T16:25:34,526   adding 'lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml'
2024-01-31T16:25:34,527   adding 'lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml'
2024-01-31T16:25:34,528   adding 'lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml'
2024-01-31T16:25:34,529   adding 'lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml'
2024-01-31T16:25:34,530   adding 'lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml'
2024-01-31T16:25:34,531   adding 'lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml'
2024-01-31T16:25:34,532   adding 'lm_eval/tasks/bigbench/generate_until/code_line_description.yaml'
2024-01-31T16:25:34,533   adding 'lm_eval/tasks/bigbench/generate_until/codenames.yaml'
2024-01-31T16:25:34,534   adding 'lm_eval/tasks/bigbench/generate_until/color.yaml'
2024-01-31T16:25:34,535   adding 'lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml'
2024-01-31T16:25:34,536   adding 'lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml'
2024-01-31T16:25:34,538   adding 'lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml'
2024-01-31T16:25:34,539   adding 'lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml'
2024-01-31T16:25:34,540   adding 'lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml'
2024-01-31T16:25:34,541   adding 'lm_eval/tasks/bigbench/generate_until/crass_ai.yaml'
2024-01-31T16:25:34,542   adding 'lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml'
2024-01-31T16:25:34,543   adding 'lm_eval/tasks/bigbench/generate_until/cryptonite.yaml'
2024-01-31T16:25:34,544   adding 'lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml'
2024-01-31T16:25:34,545   adding 'lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml'
2024-01-31T16:25:34,547   adding 'lm_eval/tasks/bigbench/generate_until/date_understanding.yaml'
2024-01-31T16:25:34,548   adding 'lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml'
2024-01-31T16:25:34,549   adding 'lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml'
2024-01-31T16:25:34,550   adding 'lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml'
2024-01-31T16:25:34,551   adding 'lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml'
2024-01-31T16:25:34,552   adding 'lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml'
2024-01-31T16:25:34,553   adding 'lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml'
2024-01-31T16:25:34,555   adding 'lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml'
2024-01-31T16:25:34,556   adding 'lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml'
2024-01-31T16:25:34,557   adding 'lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml'
2024-01-31T16:25:34,558   adding 'lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml'
2024-01-31T16:25:34,559   adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml'
2024-01-31T16:25:34,560   adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml'
2024-01-31T16:25:34,561   adding 'lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml'
2024-01-31T16:25:34,563   adding 'lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml'
2024-01-31T16:25:34,564   adding 'lm_eval/tasks/bigbench/generate_until/fact_checker.yaml'
2024-01-31T16:25:34,565   adding 'lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml'
2024-01-31T16:25:34,566   adding 'lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml'
2024-01-31T16:25:34,567   adding 'lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml'
2024-01-31T16:25:34,568   adding 'lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml'
2024-01-31T16:25:34,569   adding 'lm_eval/tasks/bigbench/generate_until/gem.yaml'
2024-01-31T16:25:34,570   adding 'lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml'
2024-01-31T16:25:34,571   adding 'lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml'
2024-01-31T16:25:34,572   adding 'lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml'
2024-01-31T16:25:34,573   adding 'lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml'
2024-01-31T16:25:34,574   adding 'lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml'
2024-01-31T16:25:34,575   adding 'lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml'
2024-01-31T16:25:34,576   adding 'lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml'
2024-01-31T16:25:34,577   adding 'lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml'
2024-01-31T16:25:34,578   adding 'lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml'
2024-01-31T16:25:34,579   adding 'lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml'
2024-01-31T16:25:34,580   adding 'lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml'
2024-01-31T16:25:34,581   adding 'lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml'
2024-01-31T16:25:34,583   adding 'lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml'
2024-01-31T16:25:34,584   adding 'lm_eval/tasks/bigbench/generate_until/implicatures.yaml'
2024-01-31T16:25:34,585   adding 'lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml'
2024-01-31T16:25:34,586   adding 'lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml'
2024-01-31T16:25:34,587   adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml'
2024-01-31T16:25:34,588   adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml'
2024-01-31T16:25:34,589   adding 'lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml'
2024-01-31T16:25:34,590   adding 'lm_eval/tasks/bigbench/generate_until/irony_identification.yaml'
2024-01-31T16:25:34,591   adding 'lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml'
2024-01-31T16:25:34,592   adding 'lm_eval/tasks/bigbench/generate_until/kannada.yaml'
2024-01-31T16:25:34,594   adding 'lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml'
2024-01-31T16:25:34,595   adding 'lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml'
2024-01-31T16:25:34,596   adding 'lm_eval/tasks/bigbench/generate_until/language_games.yaml'
2024-01-31T16:25:34,597   adding 'lm_eval/tasks/bigbench/generate_until/language_identification.yaml'
2024-01-31T16:25:34,598   adding 'lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml'
2024-01-31T16:25:34,599   adding 'lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml'
2024-01-31T16:25:34,600   adding 'lm_eval/tasks/bigbench/generate_until/list_functions.yaml'
2024-01-31T16:25:34,601   adding 'lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml'
2024-01-31T16:25:34,603   adding 'lm_eval/tasks/bigbench/generate_until/logical_args.yaml'
2024-01-31T16:25:34,604   adding 'lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml'
2024-01-31T16:25:34,605   adding 'lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml'
2024-01-31T16:25:34,606   adding 'lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml'
2024-01-31T16:25:34,607   adding 'lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml'
2024-01-31T16:25:34,608   adding 'lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml'
2024-01-31T16:25:34,609   adding 'lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml'
2024-01-31T16:25:34,610   adding 'lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml'
2024-01-31T16:25:34,611   adding 'lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml'
2024-01-31T16:25:34,612   adding 'lm_eval/tasks/bigbench/generate_until/misconceptions.yaml'
2024-01-31T16:25:34,613   adding 'lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml'
2024-01-31T16:25:34,614   adding 'lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml'
2024-01-31T16:25:34,615   adding 'lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml'
2024-01-31T16:25:34,616   adding 'lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml'
2024-01-31T16:25:34,617   adding 'lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml'
2024-01-31T16:25:34,618   adding 'lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml'
2024-01-31T16:25:34,619   adding 'lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml'
2024-01-31T16:25:34,620   adding 'lm_eval/tasks/bigbench/generate_until/multiemo.yaml'
2024-01-31T16:25:34,621   adding 'lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml'
2024-01-31T16:25:34,622   adding 'lm_eval/tasks/bigbench/generate_until/navigate.yaml'
2024-01-31T16:25:34,623   adding 'lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml'
2024-01-31T16:25:34,624   adding 'lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml'
2024-01-31T16:25:34,625   adding 'lm_eval/tasks/bigbench/generate_until/object_counting.yaml'
2024-01-31T16:25:34,626   adding 'lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml'
2024-01-31T16:25:34,627   adding 'lm_eval/tasks/bigbench/generate_until/operators.yaml'
2024-01-31T16:25:34,629   adding 'lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml'
2024-01-31T16:25:34,630   adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml'
2024-01-31T16:25:34,631   adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml'
2024-01-31T16:25:34,632   adding 'lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml'
2024-01-31T16:25:34,633   adding 'lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml'
2024-01-31T16:25:34,634   adding 'lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml'
2024-01-31T16:25:34,635   adding 'lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml'
2024-01-31T16:25:34,636   adding 'lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml'
2024-01-31T16:25:34,637   adding 'lm_eval/tasks/bigbench/generate_until/physics.yaml'
2024-01-31T16:25:34,639   adding 'lm_eval/tasks/bigbench/generate_until/physics_questions.yaml'
2024-01-31T16:25:34,640   adding 'lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml'
2024-01-31T16:25:34,641   adding 'lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml'
2024-01-31T16:25:34,642   adding 'lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml'
2024-01-31T16:25:34,643   adding 'lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml'
2024-01-31T16:25:34,644   adding 'lm_eval/tasks/bigbench/generate_until/question_selection.yaml'
2024-01-31T16:25:34,645   adding 'lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml'
2024-01-31T16:25:34,647   adding 'lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml'
2024-01-31T16:25:34,648   adding 'lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml'
2024-01-31T16:25:34,649   adding 'lm_eval/tasks/bigbench/generate_until/rephrase.yaml'
2024-01-31T16:25:34,650   adding 'lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml'
2024-01-31T16:25:34,651   adding 'lm_eval/tasks/bigbench/generate_until/ruin_names.yaml'
2024-01-31T16:25:34,652   adding 'lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml'
2024-01-31T16:25:34,653   adding 'lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml'
2024-01-31T16:25:34,654   adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml'
2024-01-31T16:25:34,655   adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml'
2024-01-31T16:25:34,656   adding 'lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml'
2024-01-31T16:25:34,657   adding 'lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml'
2024-01-31T16:25:34,658   adding 'lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml'
2024-01-31T16:25:34,659   adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml'
2024-01-31T16:25:34,660   adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml'
2024-01-31T16:25:34,661   adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml'
2024-01-31T16:25:34,662   adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml'
2024-01-31T16:25:34,663   adding 'lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml'
2024-01-31T16:25:34,664   adding 'lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml'
2024-01-31T16:25:34,665   adding 'lm_eval/tasks/bigbench/generate_until/snarks.yaml'
2024-01-31T16:25:34,666   adding 'lm_eval/tasks/bigbench/generate_until/social_iqa.yaml'
2024-01-31T16:25:34,667   adding 'lm_eval/tasks/bigbench/generate_until/social_support.yaml'
2024-01-31T16:25:34,669   adding 'lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml'
2024-01-31T16:25:34,670   adding 'lm_eval/tasks/bigbench/generate_until/strange_stories.yaml'
2024-01-31T16:25:34,671   adding 'lm_eval/tasks/bigbench/generate_until/strategyqa.yaml'
2024-01-31T16:25:34,672   adding 'lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml'
2024-01-31T16:25:34,673   adding 'lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml'
2024-01-31T16:25:34,674   adding 'lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml'
2024-01-31T16:25:34,675   adding 'lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml'
2024-01-31T16:25:34,676   adding 'lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml'
2024-01-31T16:25:34,677   adding 'lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml'
2024-01-31T16:25:34,679   adding 'lm_eval/tasks/bigbench/generate_until/tense.yaml'
2024-01-31T16:25:34,680   adding 'lm_eval/tasks/bigbench/generate_until/timedial.yaml'
2024-01-31T16:25:34,681   adding 'lm_eval/tasks/bigbench/generate_until/topical_chat.yaml'
2024-01-31T16:25:34,682   adding 'lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml'
2024-01-31T16:25:34,683   adding 'lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml'
2024-01-31T16:25:34,684   adding 'lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml'
2024-01-31T16:25:34,685   adding 'lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml'
2024-01-31T16:25:34,687   adding 'lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml'
2024-01-31T16:25:34,688   adding 'lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml'
2024-01-31T16:25:34,689   adding 'lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml'
2024-01-31T16:25:34,690   adding 'lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml'
2024-01-31T16:25:34,691   adding 'lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml'
2024-01-31T16:25:34,692   adding 'lm_eval/tasks/bigbench/generate_until/winowhy.yaml'
2024-01-31T16:25:34,693   adding 'lm_eval/tasks/bigbench/generate_until/word_sorting.yaml'
2024-01-31T16:25:34,694   adding 'lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml'
2024-01-31T16:25:34,699   adding 'lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml'
2024-01-31T16:25:34,700   adding 'lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml'
2024-01-31T16:25:34,701   adding 'lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml'
2024-01-31T16:25:34,702   adding 'lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml'
2024-01-31T16:25:34,703   adding 'lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml'
2024-01-31T16:25:34,704   adding 'lm_eval/tasks/bigbench/multiple_choice/ascii_word_recognition.yaml'
2024-01-31T16:25:34,705   adding 'lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml'
2024-01-31T16:25:34,706   adding 'lm_eval/tasks/bigbench/multiple_choice/auto_categorization.yaml'
2024-01-31T16:25:34,707   adding 'lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml'
2024-01-31T16:25:34,709   adding 'lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml'
2024-01-31T16:25:34,710   adding 'lm_eval/tasks/bigbench/multiple_choice/bridging_anaphora_resolution_barqa.yaml'
2024-01-31T16:25:34,711   adding 'lm_eval/tasks/bigbench/multiple_choice/causal_judgement.yaml'
2024-01-31T16:25:34,712   adding 'lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml'
2024-01-31T16:25:34,713   adding 'lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml'
2024-01-31T16:25:34,714   adding 'lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml'
2024-01-31T16:25:34,715   adding 'lm_eval/tasks/bigbench/multiple_choice/chess_state_tracking.yaml'
2024-01-31T16:25:34,716   adding 'lm_eval/tasks/bigbench/multiple_choice/chinese_remainder_theorem.yaml'
2024-01-31T16:25:34,717   adding 'lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml'
2024-01-31T16:25:34,718   adding 'lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml'
2024-01-31T16:25:34,719   adding 'lm_eval/tasks/bigbench/multiple_choice/codenames.yaml'
2024-01-31T16:25:34,720   adding 'lm_eval/tasks/bigbench/multiple_choice/color.yaml'
2024-01-31T16:25:34,721   adding 'lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml'
2024-01-31T16:25:34,722   adding 'lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml'
2024-01-31T16:25:34,724   adding 'lm_eval/tasks/bigbench/multiple_choice/conlang_translation.yaml'
2024-01-31T16:25:34,725   adding 'lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml'
2024-01-31T16:25:34,726   adding 'lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml'
2024-01-31T16:25:34,727   adding 'lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml'
2024-01-31T16:25:34,728   adding 'lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml'
2024-01-31T16:25:34,730   adding 'lm_eval/tasks/bigbench/multiple_choice/cryptonite.yaml'
2024-01-31T16:25:34,731   adding 'lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml'
2024-01-31T16:25:34,732   adding 'lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml'
2024-01-31T16:25:34,733   adding 'lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml'
2024-01-31T16:25:34,734   adding 'lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml'
2024-01-31T16:25:34,735   adding 'lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml'
2024-01-31T16:25:34,736   adding 'lm_eval/tasks/bigbench/multiple_choice/disfl_qa.yaml'
2024-01-31T16:25:34,737   adding 'lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml'
2024-01-31T16:25:34,738   adding 'lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml'
2024-01-31T16:25:34,740   adding 'lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml'
2024-01-31T16:25:34,741   adding 'lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml'
2024-01-31T16:25:34,742   adding 'lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml'
2024-01-31T16:25:34,743   adding 'lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml'
2024-01-31T16:25:34,744   adding 'lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml'
2024-01-31T16:25:34,745   adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml'
2024-01-31T16:25:34,746   adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml'
2024-01-31T16:25:34,747   adding 'lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml'
2024-01-31T16:25:34,749   adding 'lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml'
2024-01-31T16:25:34,750   adding 'lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml'
2024-01-31T16:25:34,751   adding 'lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml'
2024-01-31T16:25:34,752   adding 'lm_eval/tasks/bigbench/multiple_choice/few_shot_nlg.yaml'
2024-01-31T16:25:34,753   adding 'lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml'
2024-01-31T16:25:34,754   adding 'lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml'
2024-01-31T16:25:34,755   adding 'lm_eval/tasks/bigbench/multiple_choice/gem.yaml'
2024-01-31T16:25:34,756   adding 'lm_eval/tasks/bigbench/multiple_choice/gender_inclusive_sentences_german.yaml'
2024-01-31T16:25:34,757   adding 'lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml'
2024-01-31T16:25:34,758   adding 'lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml'
2024-01-31T16:25:34,759   adding 'lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml'
2024-01-31T16:25:34,760   adding 'lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml'
2024-01-31T16:25:34,761   adding 'lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml'
2024-01-31T16:25:34,762   adding 'lm_eval/tasks/bigbench/multiple_choice/hindi_question_answering.yaml'
2024-01-31T16:25:34,763   adding 'lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml'
2024-01-31T16:25:34,764   adding 'lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml'
2024-01-31T16:25:34,765   adding 'lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml'
2024-01-31T16:25:34,766   adding 'lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml'
2024-01-31T16:25:34,767   adding 'lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml'
2024-01-31T16:25:34,769   adding 'lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml'
2024-01-31T16:25:34,770   adding 'lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml'
2024-01-31T16:25:34,771   adding 'lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml'
2024-01-31T16:25:34,772   adding 'lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml'
2024-01-31T16:25:34,773   adding 'lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml'
2024-01-31T16:25:34,774   adding 'lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_transliterate.yaml'
2024-01-31T16:25:34,775   adding 'lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml'
2024-01-31T16:25:34,776   adding 'lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml'
2024-01-31T16:25:34,778   adding 'lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml'
2024-01-31T16:25:34,779   adding 'lm_eval/tasks/bigbench/multiple_choice/kannada.yaml'
2024-01-31T16:25:34,780   adding 'lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml'
2024-01-31T16:25:34,781   adding 'lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml'
2024-01-31T16:25:34,782   adding 'lm_eval/tasks/bigbench/multiple_choice/language_games.yaml'
2024-01-31T16:25:34,783   adding 'lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml'
2024-01-31T16:25:34,784   adding 'lm_eval/tasks/bigbench/multiple_choice/linguistic_mappings.yaml'
2024-01-31T16:25:34,785   adding 'lm_eval/tasks/bigbench/multiple_choice/linguistics_puzzles.yaml'
2024-01-31T16:25:34,787   adding 'lm_eval/tasks/bigbench/multiple_choice/list_functions.yaml'
2024-01-31T16:25:34,788   adding 'lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml'
2024-01-31T16:25:34,789   adding 'lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml'
2024-01-31T16:25:34,790   adding 'lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml'
2024-01-31T16:25:34,791   adding 'lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml'
2024-01-31T16:25:34,792   adding 'lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml'
2024-01-31T16:25:34,793   adding 'lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml'
2024-01-31T16:25:34,794   adding 'lm_eval/tasks/bigbench/multiple_choice/matrixshapes.yaml'
2024-01-31T16:25:34,795   adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml'
2024-01-31T16:25:34,796   adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml'
2024-01-31T16:25:34,797   adding 'lm_eval/tasks/bigbench/multiple_choice/minute_mysteries_qa.yaml'
2024-01-31T16:25:34,798   adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml'
2024-01-31T16:25:34,799   adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml'
2024-01-31T16:25:34,801   adding 'lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml'
2024-01-31T16:25:34,802   adding 'lm_eval/tasks/bigbench/multiple_choice/modified_arithmetic.yaml'
2024-01-31T16:25:34,803   adding 'lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml'
2024-01-31T16:25:34,804   adding 'lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml'
2024-01-31T16:25:34,805   adding 'lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml'
2024-01-31T16:25:34,806   adding 'lm_eval/tasks/bigbench/multiple_choice/mult_data_wrangling.yaml'
2024-01-31T16:25:34,807   adding 'lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml'
2024-01-31T16:25:34,808   adding 'lm_eval/tasks/bigbench/multiple_choice/natural_instructions.yaml'
2024-01-31T16:25:34,809   adding 'lm_eval/tasks/bigbench/multiple_choice/navigate.yaml'
2024-01-31T16:25:34,810   adding 'lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml'
2024-01-31T16:25:34,811   adding 'lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml'
2024-01-31T16:25:34,812   adding 'lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml'
2024-01-31T16:25:34,813   adding 'lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml'
2024-01-31T16:25:34,815   adding 'lm_eval/tasks/bigbench/multiple_choice/operators.yaml'
2024-01-31T16:25:34,816   adding 'lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml'
2024-01-31T16:25:34,817   adding 'lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml'
2024-01-31T16:25:34,818   adding 'lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml'
2024-01-31T16:25:34,819   adding 'lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml'
2024-01-31T16:25:34,820   adding 'lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml'
2024-01-31T16:25:34,821   adding 'lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml'
2024-01-31T16:25:34,822   adding 'lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml'
2024-01-31T16:25:34,823   adding 'lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml'
2024-01-31T16:25:34,825   adding 'lm_eval/tasks/bigbench/multiple_choice/physics.yaml'
2024-01-31T16:25:34,826   adding 'lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml'
2024-01-31T16:25:34,827   adding 'lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml'
2024-01-31T16:25:34,828   adding 'lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml'
2024-01-31T16:25:34,829   adding 'lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml'
2024-01-31T16:25:34,830   adding 'lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml'
2024-01-31T16:25:34,831   adding 'lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml'
2024-01-31T16:25:34,832   adding 'lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml'
2024-01-31T16:25:34,834   adding 'lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml'
2024-01-31T16:25:34,835   adding 'lm_eval/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml'
2024-01-31T16:25:34,836   adding 'lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml'
2024-01-31T16:25:34,837   adding 'lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml'
2024-01-31T16:25:34,838   adding 'lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml'
2024-01-31T16:25:34,839   adding 'lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml'
2024-01-31T16:25:34,840   adding 'lm_eval/tasks/bigbench/multiple_choice/scientific_press_release.yaml'
2024-01-31T16:25:34,841   adding 'lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_in_context_sparc.yaml'
2024-01-31T16:25:34,842   adding 'lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_spider.yaml'
2024-01-31T16:25:34,843   adding 'lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml'
2024-01-31T16:25:34,844   adding 'lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml'
2024-01-31T16:25:34,845   adding 'lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml'
2024-01-31T16:25:34,846   adding 'lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml'
2024-01-31T16:25:34,847   adding 'lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml'
2024-01-31T16:25:34,848   adding 'lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml'
2024-01-31T16:25:34,849   adding 'lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml'
2024-01-31T16:25:34,850   adding 'lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml'
2024-01-31T16:25:34,851   adding 'lm_eval/tasks/bigbench/multiple_choice/simple_text_editing.yaml'
2024-01-31T16:25:34,852   adding 'lm_eval/tasks/bigbench/multiple_choice/snarks.yaml'
2024-01-31T16:25:34,853   adding 'lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml'
2024-01-31T16:25:34,854   adding 'lm_eval/tasks/bigbench/multiple_choice/social_support.yaml'
2024-01-31T16:25:34,855   adding 'lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml'
2024-01-31T16:25:34,857   adding 'lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml'
2024-01-31T16:25:34,858   adding 'lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml'
2024-01-31T16:25:34,859   adding 'lm_eval/tasks/bigbench/multiple_choice/sufficient_information.yaml'
2024-01-31T16:25:34,860   adding 'lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml'
2024-01-31T16:25:34,861   adding 'lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml'
2024-01-31T16:25:34,862   adding 'lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml'
2024-01-31T16:25:34,863   adding 'lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml'
2024-01-31T16:25:34,864   adding 'lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml'
2024-01-31T16:25:34,865   adding 'lm_eval/tasks/bigbench/multiple_choice/tense.yaml'
2024-01-31T16:25:34,867   adding 'lm_eval/tasks/bigbench/multiple_choice/timedial.yaml'
2024-01-31T16:25:34,868   adding 'lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml'
2024-01-31T16:25:34,869   adding 'lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml'
2024-01-31T16:25:34,870   adding 'lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml'
2024-01-31T16:25:34,871   adding 'lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml'
2024-01-31T16:25:34,872   adding 'lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml'
2024-01-31T16:25:34,873   adding 'lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml'
2024-01-31T16:25:34,874   adding 'lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml'
2024-01-31T16:25:34,875   adding 'lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml'
2024-01-31T16:25:34,877   adding 'lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml'
2024-01-31T16:25:34,878   adding 'lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml'
2024-01-31T16:25:34,879   adding 'lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml'
2024-01-31T16:25:34,880   adding 'lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml'
2024-01-31T16:25:34,881   adding 'lm_eval/tasks/bigbench/multiple_choice/word_unscrambling.yaml'
2024-01-31T16:25:34,884   adding 'lm_eval/tasks/blimp/README.md'
2024-01-31T16:25:34,885   adding 'lm_eval/tasks/blimp/_template_yaml'
2024-01-31T16:25:34,886   adding 'lm_eval/tasks/blimp/adjunct_island.yaml'
2024-01-31T16:25:34,887   adding 'lm_eval/tasks/blimp/anaphor_gender_agreement.yaml'
2024-01-31T16:25:34,888   adding 'lm_eval/tasks/blimp/anaphor_number_agreement.yaml'
2024-01-31T16:25:34,889   adding 'lm_eval/tasks/blimp/animate_subject_passive.yaml'
2024-01-31T16:25:34,890   adding 'lm_eval/tasks/blimp/animate_subject_trans.yaml'
2024-01-31T16:25:34,891   adding 'lm_eval/tasks/blimp/causative.yaml'
2024-01-31T16:25:34,892   adding 'lm_eval/tasks/blimp/complex_NP_island.yaml'
2024-01-31T16:25:34,893   adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml'
2024-01-31T16:25:34,894   adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml'
2024-01-31T16:25:34,895   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml'
2024-01-31T16:25:34,896   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml'
2024-01-31T16:25:34,897   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml'
2024-01-31T16:25:34,898   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml'
2024-01-31T16:25:34,899   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml'
2024-01-31T16:25:34,901   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml'
2024-01-31T16:25:34,902   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml'
2024-01-31T16:25:34,903   adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml'
2024-01-31T16:25:34,904   adding 'lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml'
2024-01-31T16:25:34,905   adding 'lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml'
2024-01-31T16:25:34,906   adding 'lm_eval/tasks/blimp/drop_argument.yaml'
2024-01-31T16:25:34,907   adding 'lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml'
2024-01-31T16:25:34,908   adding 'lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml'
2024-01-31T16:25:34,909   adding 'lm_eval/tasks/blimp/existential_there_object_raising.yaml'
2024-01-31T16:25:34,910   adding 'lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml'
2024-01-31T16:25:34,911   adding 'lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml'
2024-01-31T16:25:34,913   adding 'lm_eval/tasks/blimp/existential_there_subject_raising.yaml'
2024-01-31T16:25:34,914   adding 'lm_eval/tasks/blimp/expletive_it_object_raising.yaml'
2024-01-31T16:25:34,915   adding 'lm_eval/tasks/blimp/generate_configs.py'
2024-01-31T16:25:34,916   adding 'lm_eval/tasks/blimp/inchoative.yaml'
2024-01-31T16:25:34,917   adding 'lm_eval/tasks/blimp/intransitive.yaml'
2024-01-31T16:25:34,918   adding 'lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml'
2024-01-31T16:25:34,919   adding 'lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml'
2024-01-31T16:25:34,921   adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml'
2024-01-31T16:25:34,922   adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml'
2024-01-31T16:25:34,923   adding 'lm_eval/tasks/blimp/left_branch_island_echo_question.yaml'
2024-01-31T16:25:34,924   adding 'lm_eval/tasks/blimp/left_branch_island_simple_question.yaml'
2024-01-31T16:25:34,925   adding 'lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml'
2024-01-31T16:25:34,926   adding 'lm_eval/tasks/blimp/npi_present_1.yaml'
2024-01-31T16:25:34,928   adding 'lm_eval/tasks/blimp/npi_present_2.yaml'
2024-01-31T16:25:34,929   adding 'lm_eval/tasks/blimp/only_npi_licensor_present.yaml'
2024-01-31T16:25:34,930   adding 'lm_eval/tasks/blimp/only_npi_scope.yaml'
2024-01-31T16:25:34,931   adding 'lm_eval/tasks/blimp/passive_1.yaml'
2024-01-31T16:25:34,932   adding 'lm_eval/tasks/blimp/passive_2.yaml'
2024-01-31T16:25:34,933   adding 'lm_eval/tasks/blimp/principle_A_c_command.yaml'
2024-01-31T16:25:34,934   adding 'lm_eval/tasks/blimp/principle_A_case_1.yaml'
2024-01-31T16:25:34,935   adding 'lm_eval/tasks/blimp/principle_A_case_2.yaml'
2024-01-31T16:25:34,937   adding 'lm_eval/tasks/blimp/principle_A_domain_1.yaml'
2024-01-31T16:25:34,938   adding 'lm_eval/tasks/blimp/principle_A_domain_2.yaml'
2024-01-31T16:25:34,939   adding 'lm_eval/tasks/blimp/principle_A_domain_3.yaml'
2024-01-31T16:25:34,940   adding 'lm_eval/tasks/blimp/principle_A_reconstruction.yaml'
2024-01-31T16:25:34,941   adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml'
2024-01-31T16:25:34,942   adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml'
2024-01-31T16:25:34,943   adding 'lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml'
2024-01-31T16:25:34,944   adding 'lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml'
2024-01-31T16:25:34,945   adding 'lm_eval/tasks/blimp/sentential_subject_island.yaml'
2024-01-31T16:25:34,946   adding 'lm_eval/tasks/blimp/superlative_quantifiers_1.yaml'
2024-01-31T16:25:34,947   adding 'lm_eval/tasks/blimp/superlative_quantifiers_2.yaml'
2024-01-31T16:25:34,948   adding 'lm_eval/tasks/blimp/tough_vs_raising_1.yaml'
2024-01-31T16:25:34,949   adding 'lm_eval/tasks/blimp/tough_vs_raising_2.yaml'
2024-01-31T16:25:34,950   adding 'lm_eval/tasks/blimp/transitive.yaml'
2024-01-31T16:25:34,951   adding 'lm_eval/tasks/blimp/wh_island.yaml'
2024-01-31T16:25:34,952   adding 'lm_eval/tasks/blimp/wh_questions_object_gap.yaml'
2024-01-31T16:25:34,953   adding 'lm_eval/tasks/blimp/wh_questions_subject_gap.yaml'
2024-01-31T16:25:34,954   adding 'lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml'
2024-01-31T16:25:34,955   adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml'
2024-01-31T16:25:34,956   adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml'
2024-01-31T16:25:34,958   adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml'
2024-01-31T16:25:34,959   adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml'
2024-01-31T16:25:34,962   adding 'lm_eval/tasks/ceval/README.md'
2024-01-31T16:25:34,963   adding 'lm_eval/tasks/ceval/_default_ceval_yaml'
2024-01-31T16:25:34,964   adding 'lm_eval/tasks/ceval/_generate_configs.py'
2024-01-31T16:25:34,965   adding 'lm_eval/tasks/ceval/ceval-valid_accountant.yaml'
2024-01-31T16:25:34,967   adding 'lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml'
2024-01-31T16:25:34,968   adding 'lm_eval/tasks/ceval/ceval-valid_art_studies.yaml'
2024-01-31T16:25:34,969   adding 'lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml'
2024-01-31T16:25:34,970   adding 'lm_eval/tasks/ceval/ceval-valid_business_administration.yaml'
2024-01-31T16:25:34,971   adding 'lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml'
2024-01-31T16:25:34,972   adding 'lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml'
2024-01-31T16:25:34,974   adding 'lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml'
2024-01-31T16:25:34,975   adding 'lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml'
2024-01-31T16:25:34,976   adding 'lm_eval/tasks/ceval/ceval-valid_college_economics.yaml'
2024-01-31T16:25:34,977   adding 'lm_eval/tasks/ceval/ceval-valid_college_physics.yaml'
2024-01-31T16:25:34,978   adding 'lm_eval/tasks/ceval/ceval-valid_college_programming.yaml'
2024-01-31T16:25:34,979   adding 'lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml'
2024-01-31T16:25:34,980   adding 'lm_eval/tasks/ceval/ceval-valid_computer_network.yaml'
2024-01-31T16:25:34,982   adding 'lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml'
2024-01-31T16:25:34,983   adding 'lm_eval/tasks/ceval/ceval-valid_education_science.yaml'
2024-01-31T16:25:34,984   adding 'lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml'
2024-01-31T16:25:34,985   adding 'lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml'
2024-01-31T16:25:34,986   adding 'lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml'
2024-01-31T16:25:34,988   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml'
2024-01-31T16:25:34,989   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml'
2024-01-31T16:25:34,990   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml'
2024-01-31T16:25:34,991   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml'
2024-01-31T16:25:34,992   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml'
2024-01-31T16:25:34,993   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml'
2024-01-31T16:25:34,994   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml'
2024-01-31T16:25:34,995   adding 'lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml'
2024-01-31T16:25:34,997   adding 'lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml'
2024-01-31T16:25:34,998   adding 'lm_eval/tasks/ceval/ceval-valid_law.yaml'
2024-01-31T16:25:34,999   adding 'lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml'
2024-01-31T16:25:35,000   adding 'lm_eval/tasks/ceval/ceval-valid_logic.yaml'
2024-01-31T16:25:35,001   adding 'lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml'
2024-01-31T16:25:35,002   adding 'lm_eval/tasks/ceval/ceval-valid_marxism.yaml'
2024-01-31T16:25:35,003   adding 'lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml'
2024-01-31T16:25:35,004   adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml'
2024-01-31T16:25:35,005   adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml'
2024-01-31T16:25:35,006   adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml'
2024-01-31T16:25:35,007   adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml'
2024-01-31T16:25:35,009   adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml'
2024-01-31T16:25:35,010   adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml'
2024-01-31T16:25:35,011   adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml'
2024-01-31T16:25:35,012   adding 'lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml'
2024-01-31T16:25:35,013   adding 'lm_eval/tasks/ceval/ceval-valid_operating_system.yaml'
2024-01-31T16:25:35,014   adding 'lm_eval/tasks/ceval/ceval-valid_physician.yaml'
2024-01-31T16:25:35,016   adding 'lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml'
2024-01-31T16:25:35,017   adding 'lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml'
2024-01-31T16:25:35,018   adding 'lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml'
2024-01-31T16:25:35,019   adding 'lm_eval/tasks/ceval/ceval-valid_sports_science.yaml'
2024-01-31T16:25:35,020   adding 'lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml'
2024-01-31T16:25:35,021   adding 'lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml'
2024-01-31T16:25:35,022   adding 'lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml'
2024-01-31T16:25:35,024   adding 'lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml'
2024-01-31T16:25:35,027   adding 'lm_eval/tasks/cmmlu/README.md'
2024-01-31T16:25:35,028   adding 'lm_eval/tasks/cmmlu/_default_template_yaml'
2024-01-31T16:25:35,029   adding 'lm_eval/tasks/cmmlu/_generate_configs.py'
2024-01-31T16:25:35,031   adding 'lm_eval/tasks/cmmlu/cmmlu_default_agronomy.yaml'
2024-01-31T16:25:35,032   adding 'lm_eval/tasks/cmmlu/cmmlu_default_anatomy.yaml'
2024-01-31T16:25:35,033   adding 'lm_eval/tasks/cmmlu/cmmlu_default_ancient_chinese.yaml'
2024-01-31T16:25:35,034   adding 'lm_eval/tasks/cmmlu/cmmlu_default_arts.yaml'
2024-01-31T16:25:35,035   adding 'lm_eval/tasks/cmmlu/cmmlu_default_astronomy.yaml'
2024-01-31T16:25:35,036   adding 'lm_eval/tasks/cmmlu/cmmlu_default_business_ethics.yaml'
2024-01-31T16:25:35,038   adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_civil_service_exam.yaml'
2024-01-31T16:25:35,039   adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_driving_rule.yaml'
2024-01-31T16:25:35,040   adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_food_culture.yaml'
2024-01-31T16:25:35,041   adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_foreign_policy.yaml'
2024-01-31T16:25:35,042   adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_history.yaml'
2024-01-31T16:25:35,043   adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_literature.yaml'
2024-01-31T16:25:35,045   adding 'lm_eval/tasks/cmmlu/cmmlu_default_chinese_teacher_qualification.yaml'
2024-01-31T16:25:35,046   adding 'lm_eval/tasks/cmmlu/cmmlu_default_clinical_knowledge.yaml'
2024-01-31T16:25:35,047   adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_actuarial_science.yaml'
2024-01-31T16:25:35,048   adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_education.yaml'
2024-01-31T16:25:35,049   adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_engineering_hydrology.yaml'
2024-01-31T16:25:35,050   adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_law.yaml'
2024-01-31T16:25:35,051   adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_mathematics.yaml'
2024-01-31T16:25:35,053   adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_medical_statistics.yaml'
2024-01-31T16:25:35,054   adding 'lm_eval/tasks/cmmlu/cmmlu_default_college_medicine.yaml'
2024-01-31T16:25:35,055   adding 'lm_eval/tasks/cmmlu/cmmlu_default_computer_science.yaml'
2024-01-31T16:25:35,056   adding 'lm_eval/tasks/cmmlu/cmmlu_default_computer_security.yaml'
2024-01-31T16:25:35,057   adding 'lm_eval/tasks/cmmlu/cmmlu_default_conceptual_physics.yaml'
2024-01-31T16:25:35,058   adding 'lm_eval/tasks/cmmlu/cmmlu_default_construction_project_management.yaml'
2024-01-31T16:25:35,059   adding 'lm_eval/tasks/cmmlu/cmmlu_default_economics.yaml'
2024-01-31T16:25:35,060   adding 'lm_eval/tasks/cmmlu/cmmlu_default_education.yaml'
2024-01-31T16:25:35,061   adding 'lm_eval/tasks/cmmlu/cmmlu_default_electrical_engineering.yaml'
2024-01-31T16:25:35,062   adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_chinese.yaml'
2024-01-31T16:25:35,064   adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_commonsense.yaml'
2024-01-31T16:25:35,065   adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_information_and_technology.yaml'
2024-01-31T16:25:35,066   adding 'lm_eval/tasks/cmmlu/cmmlu_default_elementary_mathematics.yaml'
2024-01-31T16:25:35,067   adding 'lm_eval/tasks/cmmlu/cmmlu_default_ethnology.yaml'
2024-01-31T16:25:35,068   adding 'lm_eval/tasks/cmmlu/cmmlu_default_food_science.yaml'
2024-01-31T16:25:35,069   adding 'lm_eval/tasks/cmmlu/cmmlu_default_genetics.yaml'
2024-01-31T16:25:35,070   adding 'lm_eval/tasks/cmmlu/cmmlu_default_global_facts.yaml'
2024-01-31T16:25:35,071   adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_biology.yaml'
2024-01-31T16:25:35,073   adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_chemistry.yaml'
2024-01-31T16:25:35,074   adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_geography.yaml'
2024-01-31T16:25:35,075   adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_mathematics.yaml'
2024-01-31T16:25:35,076   adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_physics.yaml'
2024-01-31T16:25:35,077   adding 'lm_eval/tasks/cmmlu/cmmlu_default_high_school_politics.yaml'
2024-01-31T16:25:35,078   adding 'lm_eval/tasks/cmmlu/cmmlu_default_human_sexuality.yaml'
2024-01-31T16:25:35,080   adding 'lm_eval/tasks/cmmlu/cmmlu_default_international_law.yaml'
2024-01-31T16:25:35,081   adding 'lm_eval/tasks/cmmlu/cmmlu_default_journalism.yaml'
2024-01-31T16:25:35,082   adding 'lm_eval/tasks/cmmlu/cmmlu_default_jurisprudence.yaml'
2024-01-31T16:25:35,083   adding 'lm_eval/tasks/cmmlu/cmmlu_default_legal_and_moral_basis.yaml'
2024-01-31T16:25:35,084   adding 'lm_eval/tasks/cmmlu/cmmlu_default_logical.yaml'
2024-01-31T16:25:35,086   adding 'lm_eval/tasks/cmmlu/cmmlu_default_machine_learning.yaml'
2024-01-31T16:25:35,087   adding 'lm_eval/tasks/cmmlu/cmmlu_default_management.yaml'
2024-01-31T16:25:35,088   adding 'lm_eval/tasks/cmmlu/cmmlu_default_marketing.yaml'
2024-01-31T16:25:35,089   adding 'lm_eval/tasks/cmmlu/cmmlu_default_marxist_theory.yaml'
2024-01-31T16:25:35,090   adding 'lm_eval/tasks/cmmlu/cmmlu_default_modern_chinese.yaml'
2024-01-31T16:25:35,091   adding 'lm_eval/tasks/cmmlu/cmmlu_default_nutrition.yaml'
2024-01-31T16:25:35,092   adding 'lm_eval/tasks/cmmlu/cmmlu_default_philosophy.yaml'
2024-01-31T16:25:35,094   adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_accounting.yaml'
2024-01-31T16:25:35,095   adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_law.yaml'
2024-01-31T16:25:35,096   adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_medicine.yaml'
2024-01-31T16:25:35,097   adding 'lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml'
2024-01-31T16:25:35,098   adding 'lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml'
2024-01-31T16:25:35,099   adding 'lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml'
2024-01-31T16:25:35,100   adding 'lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml'
2024-01-31T16:25:35,102   adding 'lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml'
2024-01-31T16:25:35,103   adding 'lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml'
2024-01-31T16:25:35,104   adding 'lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml'
2024-01-31T16:25:35,105   adding 'lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml'
2024-01-31T16:25:35,106   adding 'lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml'
2024-01-31T16:25:35,108   adding 'lm_eval/tasks/code_x_glue/code-text/bleu.py'
2024-01-31T16:25:35,110   adding 'lm_eval/tasks/code_x_glue/code-text/go.yaml'
2024-01-31T16:25:35,111   adding 'lm_eval/tasks/code_x_glue/code-text/java.yaml'
2024-01-31T16:25:35,112   adding 'lm_eval/tasks/code_x_glue/code-text/javascript.yaml'
2024-01-31T16:25:35,113   adding 'lm_eval/tasks/code_x_glue/code-text/php.yaml'
2024-01-31T16:25:35,114   adding 'lm_eval/tasks/code_x_glue/code-text/python.yaml'
2024-01-31T16:25:35,118   adding 'lm_eval/tasks/code_x_glue/code-text/ruby.yaml'
2024-01-31T16:25:35,119   adding 'lm_eval/tasks/code_x_glue/code-text/utils.py'
2024-01-31T16:25:35,121   adding 'lm_eval/tasks/coqa/README.md'
2024-01-31T16:25:35,122   adding 'lm_eval/tasks/coqa/default.yaml'
2024-01-31T16:25:35,123   adding 'lm_eval/tasks/coqa/utils.py'
2024-01-31T16:25:35,126   adding 'lm_eval/tasks/crows_pairs/README.md'
2024-01-31T16:25:35,127   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english.yaml'
2024-01-31T16:25:35,128   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml'
2024-01-31T16:25:35,129   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml'
2024-01-31T16:25:35,130   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml'
2024-01-31T16:25:35,131   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml'
2024-01-31T16:25:35,133   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml'
2024-01-31T16:25:35,134   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml'
2024-01-31T16:25:35,135   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml'
2024-01-31T16:25:35,136   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml'
2024-01-31T16:25:35,137   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml'
2024-01-31T16:25:35,138   adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml'
2024-01-31T16:25:35,139   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french.yaml'
2024-01-31T16:25:35,140   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml'
2024-01-31T16:25:35,142   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml'
2024-01-31T16:25:35,143   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml'
2024-01-31T16:25:35,144   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml'
2024-01-31T16:25:35,145   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml'
2024-01-31T16:25:35,146   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml'
2024-01-31T16:25:35,147   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml'
2024-01-31T16:25:35,148   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml'
2024-01-31T16:25:35,150   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml'
2024-01-31T16:25:35,151   adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml'
2024-01-31T16:25:35,152   adding 'lm_eval/tasks/crows_pairs/utils.py'
2024-01-31T16:25:35,154   adding 'lm_eval/tasks/csatqa/_default_csatqa_yaml'
2024-01-31T16:25:35,155   adding 'lm_eval/tasks/csatqa/_generate_configs.py'
2024-01-31T16:25:35,156   adding 'lm_eval/tasks/csatqa/csatqa_gr.yaml'
2024-01-31T16:25:35,157   adding 'lm_eval/tasks/csatqa/csatqa_li.yaml'
2024-01-31T16:25:35,158   adding 'lm_eval/tasks/csatqa/csatqa_rch.yaml'
2024-01-31T16:25:35,159   adding 'lm_eval/tasks/csatqa/csatqa_rcs.yaml'
2024-01-31T16:25:35,161   adding 'lm_eval/tasks/csatqa/csatqa_rcss.yaml'
2024-01-31T16:25:35,161   adding 'lm_eval/tasks/csatqa/csatqa_wr.yaml'
2024-01-31T16:25:35,163   adding 'lm_eval/tasks/csatqa/utils.py'
2024-01-31T16:25:35,164   adding 'lm_eval/tasks/drop/README.md'
2024-01-31T16:25:35,165   adding 'lm_eval/tasks/drop/default.yaml'
2024-01-31T16:25:35,167   adding 'lm_eval/tasks/drop/utils.py'
2024-01-31T16:25:35,169   adding 'lm_eval/tasks/fld/README.md'
2024-01-31T16:25:35,170   adding 'lm_eval/tasks/fld/fld_default.yaml'
2024-01-31T16:25:35,171   adding 'lm_eval/tasks/fld/fld_star.yaml'
2024-01-31T16:25:35,173   adding 'lm_eval/tasks/glue/README.md'
2024-01-31T16:25:35,175   adding 'lm_eval/tasks/glue/cola/default.yaml'
2024-01-31T16:25:35,176   adding 'lm_eval/tasks/glue/mnli/default.yaml'
2024-01-31T16:25:35,178   adding 'lm_eval/tasks/glue/mnli/mismatch.yaml'
2024-01-31T16:25:35,179   adding 'lm_eval/tasks/glue/mnli/utils.py'
2024-01-31T16:25:35,180   adding 'lm_eval/tasks/glue/mrpc/default.yaml'
2024-01-31T16:25:35,182   adding 'lm_eval/tasks/glue/qnli/default.yaml'
2024-01-31T16:25:35,183   adding 'lm_eval/tasks/glue/qqp/default.yaml'
2024-01-31T16:25:35,185   adding 'lm_eval/tasks/glue/rte/default.yaml'
2024-01-31T16:25:35,186   adding 'lm_eval/tasks/glue/sst2/default.yaml'
2024-01-31T16:25:35,188   adding 'lm_eval/tasks/glue/wnli/default.yaml'
2024-01-31T16:25:35,190   adding 'lm_eval/tasks/gsm8k/README.md'
2024-01-31T16:25:35,191   adding 'lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml'
2024-01-31T16:25:35,192   adding 'lm_eval/tasks/gsm8k/gsm8k-cot.yaml'
2024-01-31T16:25:35,194   adding 'lm_eval/tasks/gsm8k/gsm8k.yaml'
2024-01-31T16:25:35,195   adding 'lm_eval/tasks/headqa/README.md'
2024-01-31T16:25:35,197   adding 'lm_eval/tasks/headqa/headqa_en.yaml'
2024-01-31T16:25:35,198   adding 'lm_eval/tasks/headqa/headqa_es.yaml'
2024-01-31T16:25:35,200   adding 'lm_eval/tasks/hellaswag/README.md'
2024-01-31T16:25:35,201   adding 'lm_eval/tasks/hellaswag/hellaswag.yaml'
2024-01-31T16:25:35,202   adding 'lm_eval/tasks/hellaswag/utils.py'
2024-01-31T16:25:35,204   adding 'lm_eval/tasks/hendrycks_ethics/README.md'
2024-01-31T16:25:35,205   adding 'lm_eval/tasks/hendrycks_ethics/commonsense.yaml'
2024-01-31T16:25:35,206   adding 'lm_eval/tasks/hendrycks_ethics/deontology.yaml'
2024-01-31T16:25:35,207   adding 'lm_eval/tasks/hendrycks_ethics/justice.yaml'
2024-01-31T16:25:35,209   adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml'
2024-01-31T16:25:35,210   adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml'
2024-01-31T16:25:35,211   adding 'lm_eval/tasks/hendrycks_ethics/utils.py'
2024-01-31T16:25:35,212   adding 'lm_eval/tasks/hendrycks_ethics/virtue.yaml'
2024-01-31T16:25:35,214   adding 'lm_eval/tasks/ifeval/README.md'
2024-01-31T16:25:35,215   adding 'lm_eval/tasks/ifeval/ifeval.yaml'
2024-01-31T16:25:35,220   adding 'lm_eval/tasks/ifeval/instructions.py'
2024-01-31T16:25:35,222   adding 'lm_eval/tasks/ifeval/instructions_registry.py'
2024-01-31T16:25:35,225   adding 'lm_eval/tasks/ifeval/instructions_util.py'
2024-01-31T16:25:35,227   adding 'lm_eval/tasks/ifeval/utils.py'
2024-01-31T16:25:35,230   adding 'lm_eval/tasks/kmmlu/README.md'
2024-01-31T16:25:35,231   adding 'lm_eval/tasks/kmmlu/_default_kmmlu_yaml'
2024-01-31T16:25:35,232   adding 'lm_eval/tasks/kmmlu/kmmlu_accounting.yaml'
2024-01-31T16:25:35,233   adding 'lm_eval/tasks/kmmlu/kmmlu_agricultural_sciences.yaml'
2024-01-31T16:25:35,235   adding 'lm_eval/tasks/kmmlu/kmmlu_aviation_engineering_and_maintenance.yaml'
2024-01-31T16:25:35,236   adding 'lm_eval/tasks/kmmlu/kmmlu_biology.yaml'
2024-01-31T16:25:35,237   adding 'lm_eval/tasks/kmmlu/kmmlu_chemical_engineering.yaml'
2024-01-31T16:25:35,238   adding 'lm_eval/tasks/kmmlu/kmmlu_chemistry.yaml'
2024-01-31T16:25:35,239   adding 'lm_eval/tasks/kmmlu/kmmlu_civil_engineering.yaml'
2024-01-31T16:25:35,240   adding 'lm_eval/tasks/kmmlu/kmmlu_computer_science.yaml'
2024-01-31T16:25:35,241   adding 'lm_eval/tasks/kmmlu/kmmlu_construction.yaml'
2024-01-31T16:25:35,242   adding 'lm_eval/tasks/kmmlu/kmmlu_criminal_law.yaml'
2024-01-31T16:25:35,243   adding 'lm_eval/tasks/kmmlu/kmmlu_ecology.yaml'
2024-01-31T16:25:35,244   adding 'lm_eval/tasks/kmmlu/kmmlu_economics.yaml'
2024-01-31T16:25:35,245   adding 'lm_eval/tasks/kmmlu/kmmlu_education.yaml'
2024-01-31T16:25:35,246   adding 'lm_eval/tasks/kmmlu/kmmlu_electrical_engineering.yaml'
2024-01-31T16:25:35,247   adding 'lm_eval/tasks/kmmlu/kmmlu_electronics_engineering.yaml'
2024-01-31T16:25:35,248   adding 'lm_eval/tasks/kmmlu/kmmlu_energy_management.yaml'
2024-01-31T16:25:35,250   adding 'lm_eval/tasks/kmmlu/kmmlu_environmental_science.yaml'
2024-01-31T16:25:35,251   adding 'lm_eval/tasks/kmmlu/kmmlu_fashion.yaml'
2024-01-31T16:25:35,252   adding 'lm_eval/tasks/kmmlu/kmmlu_food_processing.yaml'
2024-01-31T16:25:35,253   adding 'lm_eval/tasks/kmmlu/kmmlu_gas_technology_and_engineering.yaml'
2024-01-31T16:25:35,254   adding 'lm_eval/tasks/kmmlu/kmmlu_geomatics.yaml'
2024-01-31T16:25:35,255   adding 'lm_eval/tasks/kmmlu/kmmlu_health.yaml'
2024-01-31T16:25:35,256   adding 'lm_eval/tasks/kmmlu/kmmlu_industrial_engineer.yaml'
2024-01-31T16:25:35,257   adding 'lm_eval/tasks/kmmlu/kmmlu_information_technology.yaml'
2024-01-31T16:25:35,258   adding 'lm_eval/tasks/kmmlu/kmmlu_interior_architecture_and_design.yaml'
2024-01-31T16:25:35,259   adding 'lm_eval/tasks/kmmlu/kmmlu_law.yaml'
2024-01-31T16:25:35,260   adding 'lm_eval/tasks/kmmlu/kmmlu_machine_design_and_manufacturing.yaml'
2024-01-31T16:25:35,261   adding 'lm_eval/tasks/kmmlu/kmmlu_management.yaml'
2024-01-31T16:25:35,262   adding 'lm_eval/tasks/kmmlu/kmmlu_maritime_engineering.yaml'
2024-01-31T16:25:35,263   adding 'lm_eval/tasks/kmmlu/kmmlu_marketing.yaml'
2024-01-31T16:25:35,264   adding 'lm_eval/tasks/kmmlu/kmmlu_materials_engineering.yaml'
2024-01-31T16:25:35,266   adding 'lm_eval/tasks/kmmlu/kmmlu_mechanical_engineering.yaml'
2024-01-31T16:25:35,267   adding 'lm_eval/tasks/kmmlu/kmmlu_nondestructive_testing.yaml'
2024-01-31T16:25:35,268   adding 'lm_eval/tasks/kmmlu/kmmlu_patent.yaml'
2024-01-31T16:25:35,269   adding 'lm_eval/tasks/kmmlu/kmmlu_political_science_and_sociology.yaml'
2024-01-31T16:25:35,270   adding 'lm_eval/tasks/kmmlu/kmmlu_psychology.yaml'
2024-01-31T16:25:35,271   adding 'lm_eval/tasks/kmmlu/kmmlu_public_safety.yaml'
2024-01-31T16:25:35,272   adding 'lm_eval/tasks/kmmlu/kmmlu_railway_and_automotive_engineering.yaml'
2024-01-31T16:25:35,273   adding 'lm_eval/tasks/kmmlu/kmmlu_real_estate.yaml'
2024-01-31T16:25:35,275   adding 'lm_eval/tasks/kmmlu/kmmlu_refrigerating_machinery.yaml'
2024-01-31T16:25:35,276   adding 'lm_eval/tasks/kmmlu/kmmlu_social_welfare.yaml'
2024-01-31T16:25:35,277   adding 'lm_eval/tasks/kmmlu/kmmlu_taxation.yaml'
2024-01-31T16:25:35,278   adding 'lm_eval/tasks/kmmlu/kmmlu_telecommunications_and_wireless_technology.yaml'
2024-01-31T16:25:35,280   adding 'lm_eval/tasks/kobest/README.md'
2024-01-31T16:25:35,281   adding 'lm_eval/tasks/kobest/kobest_boolq.yaml'
2024-01-31T16:25:35,282   adding 'lm_eval/tasks/kobest/kobest_copa.yaml'
2024-01-31T16:25:35,283   adding 'lm_eval/tasks/kobest/kobest_hellaswag.yaml'
2024-01-31T16:25:35,285   adding 'lm_eval/tasks/kobest/kobest_sentineg.yaml'
2024-01-31T16:25:35,286   adding 'lm_eval/tasks/kobest/kobest_wic.yaml'
2024-01-31T16:25:35,287   adding 'lm_eval/tasks/kobest/utils.py'
2024-01-31T16:25:35,289   adding 'lm_eval/tasks/lambada/README.md'
2024-01-31T16:25:35,290   adding 'lm_eval/tasks/lambada/lambada_openai.yaml'
2024-01-31T16:25:35,291   adding 'lm_eval/tasks/lambada/lambada_standard.yaml'
2024-01-31T16:25:35,293   adding 'lm_eval/tasks/lambada_cloze/README.md'
2024-01-31T16:25:35,294   adding 'lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml'
2024-01-31T16:25:35,295   adding 'lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml'
2024-01-31T16:25:35,297   adding 'lm_eval/tasks/lambada_multilingual/README.md'
2024-01-31T16:25:35,298   adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml'
2024-01-31T16:25:35,299   adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml'
2024-01-31T16:25:35,300   adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml'
2024-01-31T16:25:35,301   adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml'
2024-01-31T16:25:35,302   adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml'
2024-01-31T16:25:35,304   adding 'lm_eval/tasks/logiqa/README.md'
2024-01-31T16:25:35,305   adding 'lm_eval/tasks/logiqa/logiqa.yaml'
2024-01-31T16:25:35,306   adding 'lm_eval/tasks/logiqa/utils_logiqa.py'
2024-01-31T16:25:35,308   adding 'lm_eval/tasks/logiqa2/README.md'
2024-01-31T16:25:35,309   adding 'lm_eval/tasks/logiqa2/logieval.yaml'
2024-01-31T16:25:35,310   adding 'lm_eval/tasks/logiqa2/logiqa2.yaml'
2024-01-31T16:25:35,312   adding 'lm_eval/tasks/logiqa2/utils_logiqa2.py'
2024-01-31T16:25:35,313   adding 'lm_eval/tasks/mathqa/README.md'
2024-01-31T16:25:35,314   adding 'lm_eval/tasks/mathqa/mathqa.yaml'
2024-01-31T16:25:35,315   adding 'lm_eval/tasks/mathqa/utils.py'
2024-01-31T16:25:35,317   adding 'lm_eval/tasks/mc_taco/README.md'
2024-01-31T16:25:35,318   adding 'lm_eval/tasks/mc_taco/default.yaml'
2024-01-31T16:25:35,320   adding 'lm_eval/tasks/medmcqa/medmcqa.yaml'
2024-01-31T16:25:35,321   adding 'lm_eval/tasks/medmcqa/utils_medmcqa.py'
2024-01-31T16:25:35,322   adding 'lm_eval/tasks/medqa/medqa.yaml'
2024-01-31T16:25:35,323   adding 'lm_eval/tasks/medqa/preprocess_medqa.py'
2024-01-31T16:25:35,325   adding 'lm_eval/tasks/mgsm/README.md'
2024-01-31T16:25:35,327   adding 'lm_eval/tasks/mgsm/utils.py'
2024-01-31T16:25:35,329   adding 'lm_eval/tasks/mgsm/direct/direct_yaml'
2024-01-31T16:25:35,330   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml'
2024-01-31T16:25:35,331   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml'
2024-01-31T16:25:35,332   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml'
2024-01-31T16:25:35,333   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml'
2024-01-31T16:25:35,335   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml'
2024-01-31T16:25:35,336   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml'
2024-01-31T16:25:35,337   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml'
2024-01-31T16:25:35,338   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml'
2024-01-31T16:25:35,339   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml'
2024-01-31T16:25:35,340   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml'
2024-01-31T16:25:35,341   adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml'
2024-01-31T16:25:35,343   adding 'lm_eval/tasks/mgsm/en_cot/cot_yaml'
2024-01-31T16:25:35,344   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_bn_en-cot.yaml'
2024-01-31T16:25:35,345   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_de_en-cot.yaml'
2024-01-31T16:25:35,347   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_en-cot.yaml'
2024-01-31T16:25:35,348   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_es_en-cot.yaml'
2024-01-31T16:25:35,349   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_fr_en-cot.yaml'
2024-01-31T16:25:35,350   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_ja_en-cot.yaml'
2024-01-31T16:25:35,351   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_ru_en-cot.yaml'
2024-01-31T16:25:35,352   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_sw_en-cot.yaml'
2024-01-31T16:25:35,354   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_te_en-cot.yaml'
2024-01-31T16:25:35,355   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_th_en-cot.yaml'
2024-01-31T16:25:35,356   adding 'lm_eval/tasks/mgsm/en_cot/mgsm_zh_en-cot.yaml'
2024-01-31T16:25:35,358   adding 'lm_eval/tasks/mgsm/native_cot/cot_yaml'
2024-01-31T16:25:35,359   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_bn.yaml'
2024-01-31T16:25:35,360   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_de.yaml'
2024-01-31T16:25:35,361   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_en.yaml'
2024-01-31T16:25:35,362   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_es.yaml'
2024-01-31T16:25:35,363   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_fr.yaml'
2024-01-31T16:25:35,364   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_ja.yaml'
2024-01-31T16:25:35,365   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_ru.yaml'
2024-01-31T16:25:35,368   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_sw.yaml'
2024-01-31T16:25:35,369   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_te.yaml'
2024-01-31T16:25:35,369   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_th.yaml'
2024-01-31T16:25:35,370   adding 'lm_eval/tasks/mgsm/native_cot/mgsm_cot_native_zh.yaml'
2024-01-31T16:25:35,371   adding 'lm_eval/tasks/minerva_math/README.md'
2024-01-31T16:25:35,372   adding 'lm_eval/tasks/minerva_math/minerva_math_algebra.yaml'
2024-01-31T16:25:35,373   adding 'lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml'
2024-01-31T16:25:35,374   adding 'lm_eval/tasks/minerva_math/minerva_math_geometry.yaml'
2024-01-31T16:25:35,375   adding 'lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml'
2024-01-31T16:25:35,376   adding 'lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml'
2024-01-31T16:25:35,377   adding 'lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml'
2024-01-31T16:25:35,378   adding 'lm_eval/tasks/minerva_math/minerva_math_precalc.yaml'
2024-01-31T16:25:35,380   adding 'lm_eval/tasks/minerva_math/utils.py'
2024-01-31T16:25:35,382   adding 'lm_eval/tasks/mmlu/_generate_configs.py'
2024-01-31T16:25:35,385   adding 'lm_eval/tasks/mmlu/default/_default_template_yaml'
2024-01-31T16:25:35,386   adding 'lm_eval/tasks/mmlu/default/_mmlu.yaml'
2024-01-31T16:25:35,387   adding 'lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml'
2024-01-31T16:25:35,388   adding 'lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml'
2024-01-31T16:25:35,389   adding 'lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml'
2024-01-31T16:25:35,390   adding 'lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml'
2024-01-31T16:25:35,391   adding 'lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml'
2024-01-31T16:25:35,393   adding 'lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml'
2024-01-31T16:25:35,394   adding 'lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml'
2024-01-31T16:25:35,395   adding 'lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml'
2024-01-31T16:25:35,396   adding 'lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml'
2024-01-31T16:25:35,397   adding 'lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml'
2024-01-31T16:25:35,398   adding 'lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml'
2024-01-31T16:25:35,400   adding 'lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml'
2024-01-31T16:25:35,401   adding 'lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml'
2024-01-31T16:25:35,402   adding 'lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml'
2024-01-31T16:25:35,403   adding 'lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml'
2024-01-31T16:25:35,404   adding 'lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml'
2024-01-31T16:25:35,405   adding 'lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml'
2024-01-31T16:25:35,406   adding 'lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml'
2024-01-31T16:25:35,408   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml'
2024-01-31T16:25:35,409   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml'
2024-01-31T16:25:35,410   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml'
2024-01-31T16:25:35,411   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml'
2024-01-31T16:25:35,412   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml'
2024-01-31T16:25:35,413   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml'
2024-01-31T16:25:35,415   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml'
2024-01-31T16:25:35,416   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml'
2024-01-31T16:25:35,417   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml'
2024-01-31T16:25:35,418   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml'
2024-01-31T16:25:35,419   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml'
2024-01-31T16:25:35,420   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml'
2024-01-31T16:25:35,421   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml'
2024-01-31T16:25:35,422   adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml'
2024-01-31T16:25:35,423   adding 'lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml'
2024-01-31T16:25:35,425   adding 'lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml'
2024-01-31T16:25:35,426   adding 'lm_eval/tasks/mmlu/default/mmlu_international_law.yaml'
2024-01-31T16:25:35,427   adding 'lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml'
2024-01-31T16:25:35,428   adding 'lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml'
2024-01-31T16:25:35,429   adding 'lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml'
2024-01-31T16:25:35,430   adding 'lm_eval/tasks/mmlu/default/mmlu_management.yaml'
2024-01-31T16:25:35,431   adding 'lm_eval/tasks/mmlu/default/mmlu_marketing.yaml'
2024-01-31T16:25:35,432   adding 'lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml'
2024-01-31T16:25:35,433   adding 'lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml'
2024-01-31T16:25:35,434   adding 'lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml'
2024-01-31T16:25:35,436   adding 'lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml'
2024-01-31T16:25:35,437   adding 'lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml'
2024-01-31T16:25:35,438   adding 'lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml'
2024-01-31T16:25:35,439   adding 'lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml'
2024-01-31T16:25:35,440   adding 'lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml'
2024-01-31T16:25:35,441   adding 'lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml'
2024-01-31T16:25:35,442   adding 'lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml'
2024-01-31T16:25:35,444   adding 'lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml'
2024-01-31T16:25:35,445   adding 'lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml'
2024-01-31T16:25:35,446   adding 'lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml'
2024-01-31T16:25:35,447   adding 'lm_eval/tasks/mmlu/default/mmlu_sociology.yaml'
2024-01-31T16:25:35,448   adding 'lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml'
2024-01-31T16:25:35,449   adding 'lm_eval/tasks/mmlu/default/mmlu_virology.yaml'
2024-01-31T16:25:35,450   adding 'lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml'
2024-01-31T16:25:35,490   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json'
2024-01-31T16:25:35,493   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml'
2024-01-31T16:25:35,494   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml'
2024-01-31T16:25:35,495   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml'
2024-01-31T16:25:35,497   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml'
2024-01-31T16:25:35,499   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml'
2024-01-31T16:25:35,500   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml'
2024-01-31T16:25:35,502   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml'
2024-01-31T16:25:35,503   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml'
2024-01-31T16:25:35,504   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml'
2024-01-31T16:25:35,506   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml'
2024-01-31T16:25:35,508   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml'
2024-01-31T16:25:35,509   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml'
2024-01-31T16:25:35,510   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml'
2024-01-31T16:25:35,512   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml'
2024-01-31T16:25:35,513   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml'
2024-01-31T16:25:35,515   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml'
2024-01-31T16:25:35,516   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml'
2024-01-31T16:25:35,518   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml'
2024-01-31T16:25:35,519   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml'
2024-01-31T16:25:35,521   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml'
2024-01-31T16:25:35,522   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml'
2024-01-31T16:25:35,523   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml'
2024-01-31T16:25:35,525   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml'
2024-01-31T16:25:35,528   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml'
2024-01-31T16:25:35,529   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml'
2024-01-31T16:25:35,531   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml'
2024-01-31T16:25:35,532   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml'
2024-01-31T16:25:35,533   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml'
2024-01-31T16:25:35,535   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml'
2024-01-31T16:25:35,536   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml'
2024-01-31T16:25:35,537   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml'
2024-01-31T16:25:35,539   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml'
2024-01-31T16:25:35,541   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml'
2024-01-31T16:25:35,543   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml'
2024-01-31T16:25:35,545   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml'
2024-01-31T16:25:35,546   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml'
2024-01-31T16:25:35,547   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml'
2024-01-31T16:25:35,549   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml'
2024-01-31T16:25:35,550   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml'
2024-01-31T16:25:35,552   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml'
2024-01-31T16:25:35,553   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml'
2024-01-31T16:25:35,555   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml'
2024-01-31T16:25:35,556   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml'
2024-01-31T16:25:35,557   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml'
2024-01-31T16:25:35,559   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml'
2024-01-31T16:25:35,560   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml'
2024-01-31T16:25:35,562   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml'
2024-01-31T16:25:35,563   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml'
2024-01-31T16:25:35,564   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml'
2024-01-31T16:25:35,566   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml'
2024-01-31T16:25:35,568   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml'
2024-01-31T16:25:35,570   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml'
2024-01-31T16:25:35,571   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml'
2024-01-31T16:25:35,573   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml'
2024-01-31T16:25:35,575   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml'
2024-01-31T16:25:35,576   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml'
2024-01-31T16:25:35,577   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml'
2024-01-31T16:25:35,579   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml'
2024-01-31T16:25:35,580   adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml'
2024-01-31T16:25:35,582   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml'
2024-01-31T16:25:35,583   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml'
2024-01-31T16:25:35,584   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml'
2024-01-31T16:25:35,585   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml'
2024-01-31T16:25:35,587   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml'
2024-01-31T16:25:35,588   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml'
2024-01-31T16:25:35,589   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml'
2024-01-31T16:25:35,590   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml'
2024-01-31T16:25:35,591   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml'
2024-01-31T16:25:35,592   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml'
2024-01-31T16:25:35,593   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml'
2024-01-31T16:25:35,595   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml'
2024-01-31T16:25:35,596   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml'
2024-01-31T16:25:35,597   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml'
2024-01-31T16:25:35,598   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml'
2024-01-31T16:25:35,599   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml'
2024-01-31T16:25:35,600   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml'
2024-01-31T16:25:35,602   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml'
2024-01-31T16:25:35,603   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml'
2024-01-31T16:25:35,604   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml'
2024-01-31T16:25:35,606   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml'
2024-01-31T16:25:35,607   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml'
2024-01-31T16:25:35,608   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml'
2024-01-31T16:25:35,609   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml'
2024-01-31T16:25:35,610   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml'
2024-01-31T16:25:35,611   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml'
2024-01-31T16:25:35,613   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml'
2024-01-31T16:25:35,614   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml'
2024-01-31T16:25:35,615   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml'
2024-01-31T16:25:35,616   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml'
2024-01-31T16:25:35,617   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml'
2024-01-31T16:25:35,619   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml'
2024-01-31T16:25:35,620   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml'
2024-01-31T16:25:35,621   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml'
2024-01-31T16:25:35,622   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml'
2024-01-31T16:25:35,623   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml'
2024-01-31T16:25:35,624   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml'
2024-01-31T16:25:35,625   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml'
2024-01-31T16:25:35,626   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml'
2024-01-31T16:25:35,627   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml'
2024-01-31T16:25:35,629   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml'
2024-01-31T16:25:35,630   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml'
2024-01-31T16:25:35,631   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml'
2024-01-31T16:25:35,632   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml'
2024-01-31T16:25:35,633   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml'
2024-01-31T16:25:35,634   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml'
2024-01-31T16:25:35,635   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml'
2024-01-31T16:25:35,637   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml'
2024-01-31T16:25:35,638   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml'
2024-01-31T16:25:35,639   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml'
2024-01-31T16:25:35,640   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml'
2024-01-31T16:25:35,641   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml'
2024-01-31T16:25:35,642   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml'
2024-01-31T16:25:35,644   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml'
2024-01-31T16:25:35,645   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml'
2024-01-31T16:25:35,646   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml'
2024-01-31T16:25:35,647   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml'
2024-01-31T16:25:35,648   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml'
2024-01-31T16:25:35,649   adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml'
2024-01-31T16:25:35,652   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml'
2024-01-31T16:25:35,653   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml'
2024-01-31T16:25:35,654   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml'
2024-01-31T16:25:35,656   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml'
2024-01-31T16:25:35,657   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml'
2024-01-31T16:25:35,658   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml'
2024-01-31T16:25:35,659   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml'
2024-01-31T16:25:35,660   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml'
2024-01-31T16:25:35,662   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml'
2024-01-31T16:25:35,663   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml'
2024-01-31T16:25:35,664   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml'
2024-01-31T16:25:35,665   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml'
2024-01-31T16:25:35,666   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml'
2024-01-31T16:25:35,667   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml'
2024-01-31T16:25:35,669   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml'
2024-01-31T16:25:35,670   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml'
2024-01-31T16:25:35,671   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml'
2024-01-31T16:25:35,672   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml'
2024-01-31T16:25:35,673   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml'
2024-01-31T16:25:35,675   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml'
2024-01-31T16:25:35,676   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml'
2024-01-31T16:25:35,677   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml'
2024-01-31T16:25:35,678   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml'
2024-01-31T16:25:35,679   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml'
2024-01-31T16:25:35,680   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml'
2024-01-31T16:25:35,681   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml'
2024-01-31T16:25:35,682   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml'
2024-01-31T16:25:35,683   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml'
2024-01-31T16:25:35,685   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml'
2024-01-31T16:25:35,686   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml'
2024-01-31T16:25:35,687   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml'
2024-01-31T16:25:35,688   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml'
2024-01-31T16:25:35,689   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml'
2024-01-31T16:25:35,690   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml'
2024-01-31T16:25:35,691   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml'
2024-01-31T16:25:35,693   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml'
2024-01-31T16:25:35,694   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml'
2024-01-31T16:25:35,695   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml'
2024-01-31T16:25:35,696   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml'
2024-01-31T16:25:35,697   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml'
2024-01-31T16:25:35,698   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml'
2024-01-31T16:25:35,699   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml'
2024-01-31T16:25:35,700   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml'
2024-01-31T16:25:35,702   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml'
2024-01-31T16:25:35,703   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml'
2024-01-31T16:25:35,704   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml'
2024-01-31T16:25:35,705   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml'
2024-01-31T16:25:35,706   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml'
2024-01-31T16:25:35,707   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml'
2024-01-31T16:25:35,708   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml'
2024-01-31T16:25:35,710   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml'
2024-01-31T16:25:35,711   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml'
2024-01-31T16:25:35,712   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml'
2024-01-31T16:25:35,713   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml'
2024-01-31T16:25:35,714   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml'
2024-01-31T16:25:35,715   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml'
2024-01-31T16:25:35,716   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml'
2024-01-31T16:25:35,717   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml'
2024-01-31T16:25:35,719   adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml'
2024-01-31T16:25:35,721   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml'
2024-01-31T16:25:35,722   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml'
2024-01-31T16:25:35,723   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml'
2024-01-31T16:25:35,724   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml'
2024-01-31T16:25:35,725   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml'
2024-01-31T16:25:35,726   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml'
2024-01-31T16:25:35,728   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml'
2024-01-31T16:25:35,729   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml'
2024-01-31T16:25:35,730   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml'
2024-01-31T16:25:35,731   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml'
2024-01-31T16:25:35,732   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml'
2024-01-31T16:25:35,733   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml'
2024-01-31T16:25:35,734   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml'
2024-01-31T16:25:35,736   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml'
2024-01-31T16:25:35,737   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml'
2024-01-31T16:25:35,738   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml'
2024-01-31T16:25:35,739   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml'
2024-01-31T16:25:35,740   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml'
2024-01-31T16:25:35,741   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml'
2024-01-31T16:25:35,743   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml'
2024-01-31T16:25:35,744   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml'
2024-01-31T16:25:35,745   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml'
2024-01-31T16:25:35,746   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml'
2024-01-31T16:25:35,747   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml'
2024-01-31T16:25:35,748   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml'
2024-01-31T16:25:35,750   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml'
2024-01-31T16:25:35,751   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml'
2024-01-31T16:25:35,752   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml'
2024-01-31T16:25:35,753   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml'
2024-01-31T16:25:35,754   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml'
2024-01-31T16:25:35,755   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml'
2024-01-31T16:25:35,757   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml'
2024-01-31T16:25:35,758   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml'
2024-01-31T16:25:35,759   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml'
2024-01-31T16:25:35,760   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml'
2024-01-31T16:25:35,761   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml'
2024-01-31T16:25:35,762   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml'
2024-01-31T16:25:35,763   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml'
2024-01-31T16:25:35,764   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml'
2024-01-31T16:25:35,765   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml'
2024-01-31T16:25:35,766   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml'
2024-01-31T16:25:35,767   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml'
2024-01-31T16:25:35,769   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml'
2024-01-31T16:25:35,770   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml'
2024-01-31T16:25:35,771   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml'
2024-01-31T16:25:35,772   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml'
2024-01-31T16:25:35,773   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml'
2024-01-31T16:25:35,774   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml'
2024-01-31T16:25:35,776   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml'
2024-01-31T16:25:35,777   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml'
2024-01-31T16:25:35,778   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml'
2024-01-31T16:25:35,779   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml'
2024-01-31T16:25:35,780   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml'
2024-01-31T16:25:35,781   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml'
2024-01-31T16:25:35,782   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml'
2024-01-31T16:25:35,784   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml'
2024-01-31T16:25:35,785   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml'
2024-01-31T16:25:35,786   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml'
2024-01-31T16:25:35,787   adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml'
2024-01-31T16:25:35,790   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py'
2024-01-31T16:25:35,791   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml'
2024-01-31T16:25:35,792   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml'
2024-01-31T16:25:35,793   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml'
2024-01-31T16:25:35,795   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml'
2024-01-31T16:25:35,796   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml'
2024-01-31T16:25:35,797   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml'
2024-01-31T16:25:35,798   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml'
2024-01-31T16:25:35,800   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml'
2024-01-31T16:25:35,801   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml'
2024-01-31T16:25:35,802   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml'
2024-01-31T16:25:35,803   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml'
2024-01-31T16:25:35,804   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml'
2024-01-31T16:25:35,805   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml'
2024-01-31T16:25:35,806   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml'
2024-01-31T16:25:35,808   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml'
2024-01-31T16:25:35,809   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml'
2024-01-31T16:25:35,810   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml'
2024-01-31T16:25:35,811   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml'
2024-01-31T16:25:35,812   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml'
2024-01-31T16:25:35,813   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml'
2024-01-31T16:25:35,814   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml'
2024-01-31T16:25:35,815   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml'
2024-01-31T16:25:35,816   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml'
2024-01-31T16:25:35,817   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml'
2024-01-31T16:25:35,819   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml'
2024-01-31T16:25:35,820   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml'
2024-01-31T16:25:35,821   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml'
2024-01-31T16:25:35,822   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml'
2024-01-31T16:25:35,823   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml'
2024-01-31T16:25:35,824   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml'
2024-01-31T16:25:35,825   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml'
2024-01-31T16:25:35,826   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml'
2024-01-31T16:25:35,828   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml'
2024-01-31T16:25:35,829   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml'
2024-01-31T16:25:35,830   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml'
2024-01-31T16:25:35,831   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml'
2024-01-31T16:25:35,832   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml'
2024-01-31T16:25:35,833   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml'
2024-01-31T16:25:35,834   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml'
2024-01-31T16:25:35,836   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml'
2024-01-31T16:25:35,837   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml'
2024-01-31T16:25:35,838   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml'
2024-01-31T16:25:35,839   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml'
2024-01-31T16:25:35,840   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml'
2024-01-31T16:25:35,841   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml'
2024-01-31T16:25:35,842   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml'
2024-01-31T16:25:35,844   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml'
2024-01-31T16:25:35,845   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml'
2024-01-31T16:25:35,846   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml'
2024-01-31T16:25:35,847   adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml'
2024-01-31T16:25:35,851   adding 'lm_eval/tasks/model_written_evals/persona/_generate_configs.py'
2024-01-31T16:25:35,852   adding 'lm_eval/tasks/model_written_evals/persona/_template_yaml'
2024-01-31T16:25:35,853   adding 'lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml'
2024-01-31T16:25:35,854   adding 'lm_eval/tasks/model_written_evals/persona/agreeableness.yaml'
2024-01-31T16:25:35,855   adding 'lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml'
2024-01-31T16:25:35,856   adding 'lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml'
2024-01-31T16:25:35,858   adding 'lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml'
2024-01-31T16:25:35,859   adding 'lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml'
2024-01-31T16:25:35,860   adding 'lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml'
2024-01-31T16:25:35,861   adding 'lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml'
2024-01-31T16:25:35,862   adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml'
2024-01-31T16:25:35,863   adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml'
2024-01-31T16:25:35,864   adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml'
2024-01-31T16:25:35,865   adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml'
2024-01-31T16:25:35,867   adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml'
2024-01-31T16:25:35,868   adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml'
2024-01-31T16:25:35,869   adding 'lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml'
2024-01-31T16:25:35,870   adding 'lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml'
2024-01-31T16:25:35,871   adding 'lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml'
2024-01-31T16:25:35,872   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml'
2024-01-31T16:25:35,873   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml'
2024-01-31T16:25:35,875   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml'
2024-01-31T16:25:35,876   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml'
2024-01-31T16:25:35,877   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml'
2024-01-31T16:25:35,878   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml'
2024-01-31T16:25:35,879   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml'
2024-01-31T16:25:35,880   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml'
2024-01-31T16:25:35,882   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml'
2024-01-31T16:25:35,883   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml'
2024-01-31T16:25:35,884   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml'
2024-01-31T16:25:35,885   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml'
2024-01-31T16:25:35,886   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml'
2024-01-31T16:25:35,887   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml'
2024-01-31T16:25:35,888   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml'
2024-01-31T16:25:35,889   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml'
2024-01-31T16:25:35,890   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml'
2024-01-31T16:25:35,891   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml'
2024-01-31T16:25:35,892   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml'
2024-01-31T16:25:35,893   adding 'lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml'
2024-01-31T16:25:35,894   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml'
2024-01-31T16:25:35,896   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml'
2024-01-31T16:25:35,897   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml'
2024-01-31T16:25:35,898   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml'
2024-01-31T16:25:35,899   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml'
2024-01-31T16:25:35,900   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml'
2024-01-31T16:25:35,901   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml'
2024-01-31T16:25:35,902   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml'
2024-01-31T16:25:35,903   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml'
2024-01-31T16:25:35,905   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml'
2024-01-31T16:25:35,906   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml'
2024-01-31T16:25:35,907   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml'
2024-01-31T16:25:35,908   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml'
2024-01-31T16:25:35,909   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml'
2024-01-31T16:25:35,910   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml'
2024-01-31T16:25:35,911   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml'
2024-01-31T16:25:35,912   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml'
2024-01-31T16:25:35,914   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml'
2024-01-31T16:25:35,915   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml'
2024-01-31T16:25:35,916   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml'
2024-01-31T16:25:35,917   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml'
2024-01-31T16:25:35,918   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml'
2024-01-31T16:25:35,919   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml'
2024-01-31T16:25:35,920   adding 'lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml'
2024-01-31T16:25:35,921   adding 'lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml'
2024-01-31T16:25:35,923   adding 'lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml'
2024-01-31T16:25:35,924   adding 'lm_eval/tasks/model_written_evals/persona/extraversion.yaml'
2024-01-31T16:25:35,925   adding 'lm_eval/tasks/model_written_evals/persona/has-disability.yaml'
2024-01-31T16:25:35,926   adding 'lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml'
2024-01-31T16:25:35,927   adding 'lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml'
2024-01-31T16:25:35,928   adding 'lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml'
2024-01-31T16:25:35,929   adding 'lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml'
2024-01-31T16:25:35,930   adding 'lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml'
2024-01-31T16:25:35,931   adding 'lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml'
2024-01-31T16:25:35,932   adding 'lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml'
2024-01-31T16:25:35,933   adding 'lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml'
2024-01-31T16:25:35,934   adding 'lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml'
2024-01-31T16:25:35,935   adding 'lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml'
2024-01-31T16:25:35,936   adding 'lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml'
2024-01-31T16:25:35,937   adding 'lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml'
2024-01-31T16:25:35,938   adding 'lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml'
2024-01-31T16:25:35,939   adding 'lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml'
2024-01-31T16:25:35,940   adding 'lm_eval/tasks/model_written_evals/persona/narcissism.yaml'
2024-01-31T16:25:35,941   adding 'lm_eval/tasks/model_written_evals/persona/neuroticism.yaml'
2024-01-31T16:25:35,942   adding 'lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml'
2024-01-31T16:25:35,943   adding 'lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml'
2024-01-31T16:25:35,944   adding 'lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml'
2024-01-31T16:25:35,946   adding 'lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml'
2024-01-31T16:25:35,947   adding 'lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml'
2024-01-31T16:25:35,948   adding 'lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml'
2024-01-31T16:25:35,949   adding 'lm_eval/tasks/model_written_evals/persona/openness.yaml'
2024-01-31T16:25:35,950   adding 'lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml'
2024-01-31T16:25:35,951   adding 'lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml'
2024-01-31T16:25:35,952   adding 'lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml'
2024-01-31T16:25:35,953   adding 'lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml'
2024-01-31T16:25:35,954   adding 'lm_eval/tasks/model_written_evals/persona/psychopathy.yaml'
2024-01-31T16:25:35,956   adding 'lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml'
2024-01-31T16:25:35,957   adding 'lm_eval/tasks/model_written_evals/persona/risk-averse.yaml'
2024-01-31T16:25:35,958   adding 'lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml'
2024-01-31T16:25:35,959   adding 'lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml'
2024-01-31T16:25:35,960   adding 'lm_eval/tasks/model_written_evals/persona/self-replication.yaml'
2024-01-31T16:25:35,961   adding 'lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml'
2024-01-31T16:25:35,962   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml'
2024-01-31T16:25:35,963   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml'
2024-01-31T16:25:35,964   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml'
2024-01-31T16:25:35,966   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml'
2024-01-31T16:25:35,967   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml'
2024-01-31T16:25:35,967   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml'
2024-01-31T16:25:35,969   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml'
2024-01-31T16:25:35,970   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml'
2024-01-31T16:25:35,971   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml'
2024-01-31T16:25:35,972   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml'
2024-01-31T16:25:35,973   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml'
2024-01-31T16:25:35,974   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml'
2024-01-31T16:25:35,975   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml'
2024-01-31T16:25:35,976   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml'
2024-01-31T16:25:35,977   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml'
2024-01-31T16:25:35,978   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml'
2024-01-31T16:25:35,979   adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml'
2024-01-31T16:25:35,980   adding 'lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml'
2024-01-31T16:25:35,982   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml'
2024-01-31T16:25:35,983   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml'
2024-01-31T16:25:35,984   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml'
2024-01-31T16:25:35,985   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml'
2024-01-31T16:25:35,986   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml'
2024-01-31T16:25:35,987   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml'
2024-01-31T16:25:35,988   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml'
2024-01-31T16:25:35,989   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml'
2024-01-31T16:25:35,990   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml'
2024-01-31T16:25:35,991   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml'
2024-01-31T16:25:35,993   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml'
2024-01-31T16:25:35,994   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml'
2024-01-31T16:25:35,995   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml'
2024-01-31T16:25:35,996   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml'
2024-01-31T16:25:35,997   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml'
2024-01-31T16:25:35,999   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml'
2024-01-31T16:25:36,000   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml'
2024-01-31T16:25:36,001   adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml'
2024-01-31T16:25:36,003   adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml'
2024-01-31T16:25:36,004   adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml'
2024-01-31T16:25:36,005   adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml'
2024-01-31T16:25:36,006   adding 'lm_eval/tasks/model_written_evals/winogenerated/_template_yaml'
2024-01-31T16:25:36,008   adding 'lm_eval/tasks/mutual/README.md'
2024-01-31T16:25:36,009   adding 'lm_eval/tasks/mutual/multual_plus.yaml'
2024-01-31T16:25:36,010   adding 'lm_eval/tasks/mutual/mutual.yaml'
2024-01-31T16:25:36,011   adding 'lm_eval/tasks/mutual/utils.py'
2024-01-31T16:25:36,013   adding 'lm_eval/tasks/nq_open/README.md'
2024-01-31T16:25:36,014   adding 'lm_eval/tasks/nq_open/nq_open.yaml'
2024-01-31T16:25:36,017   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/README.md'
2024-01-31T16:25:36,018   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml'
2024-01-31T16:25:36,019   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml'
2024-01-31T16:25:36,020   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml'
2024-01-31T16:25:36,021   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml'
2024-01-31T16:25:36,022   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml'
2024-01-31T16:25:36,023   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml'
2024-01-31T16:25:36,024   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml'
2024-01-31T16:25:36,025   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml'
2024-01-31T16:25:36,026   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml'
2024-01-31T16:25:36,028   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml'
2024-01-31T16:25:36,029   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml'
2024-01-31T16:25:36,030   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml'
2024-01-31T16:25:36,031   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml'
2024-01-31T16:25:36,032   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml'
2024-01-31T16:25:36,033   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml'
2024-01-31T16:25:36,034   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml'
2024-01-31T16:25:36,035   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml'
2024-01-31T16:25:36,036   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml'
2024-01-31T16:25:36,038   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml'
2024-01-31T16:25:36,039   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml'
2024-01-31T16:25:36,040   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml'
2024-01-31T16:25:36,041   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml'
2024-01-31T16:25:36,042   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml'
2024-01-31T16:25:36,043   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml'
2024-01-31T16:25:36,044   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml'
2024-01-31T16:25:36,046   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml'
2024-01-31T16:25:36,047   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml'
2024-01-31T16:25:36,048   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml'
2024-01-31T16:25:36,049   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml'
2024-01-31T16:25:36,050   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml'
2024-01-31T16:25:36,051   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml'
2024-01-31T16:25:36,053   adding 'lm_eval/tasks/okapi/hellaswag_multilingual/utils.py'
2024-01-31T16:25:36,054   adding 'lm_eval/tasks/openbookqa/README.md'
2024-01-31T16:25:36,055   adding 'lm_eval/tasks/openbookqa/openbookqa.yaml'
2024-01-31T16:25:36,057   adding 'lm_eval/tasks/paws-x/README.md'
2024-01-31T16:25:36,058   adding 'lm_eval/tasks/paws-x/_generate_config.py'
2024-01-31T16:25:36,059   adding 'lm_eval/tasks/paws-x/paws_de.yaml'
2024-01-31T16:25:36,060   adding 'lm_eval/tasks/paws-x/paws_en.yaml'
2024-01-31T16:25:36,061   adding 'lm_eval/tasks/paws-x/paws_es.yaml'
2024-01-31T16:25:36,062   adding 'lm_eval/tasks/paws-x/paws_fr.yaml'
2024-01-31T16:25:36,063   adding 'lm_eval/tasks/paws-x/paws_ja.yaml'
2024-01-31T16:25:36,064   adding 'lm_eval/tasks/paws-x/paws_ko.yaml'
2024-01-31T16:25:36,065   adding 'lm_eval/tasks/paws-x/paws_zh.yaml'
2024-01-31T16:25:36,067   adding 'lm_eval/tasks/paws-x/pawsx_template_yaml'
2024-01-31T16:25:36,069   adding 'lm_eval/tasks/pile/README.md'
2024-01-31T16:25:36,070   adding 'lm_eval/tasks/pile/pile_arxiv.yaml'
2024-01-31T16:25:36,071   adding 'lm_eval/tasks/pile/pile_bookcorpus2.yaml'
2024-01-31T16:25:36,072   adding 'lm_eval/tasks/pile/pile_books3.yaml'
2024-01-31T16:25:36,073   adding 'lm_eval/tasks/pile/pile_dm-mathematics.yaml'
2024-01-31T16:25:36,074   adding 'lm_eval/tasks/pile/pile_enron.yaml'
2024-01-31T16:25:36,075   adding 'lm_eval/tasks/pile/pile_europarl.yaml'
2024-01-31T16:25:36,076   adding 'lm_eval/tasks/pile/pile_freelaw.yaml'
2024-01-31T16:25:36,077   adding 'lm_eval/tasks/pile/pile_github.yaml'
2024-01-31T16:25:36,078   adding 'lm_eval/tasks/pile/pile_gutenberg.yaml'
2024-01-31T16:25:36,079   adding 'lm_eval/tasks/pile/pile_hackernews.yaml'
2024-01-31T16:25:36,080   adding 'lm_eval/tasks/pile/pile_nih-exporter.yaml'
2024-01-31T16:25:36,081   adding 'lm_eval/tasks/pile/pile_opensubtitles.yaml'
2024-01-31T16:25:36,082   adding 'lm_eval/tasks/pile/pile_openwebtext2.yaml'
2024-01-31T16:25:36,083   adding 'lm_eval/tasks/pile/pile_philpapers.yaml'
2024-01-31T16:25:36,084   adding 'lm_eval/tasks/pile/pile_pile-cc.yaml'
2024-01-31T16:25:36,085   adding 'lm_eval/tasks/pile/pile_pubmed-abstracts.yaml'
2024-01-31T16:25:36,087   adding 'lm_eval/tasks/pile/pile_pubmed-central.yaml'
2024-01-31T16:25:36,088   adding 'lm_eval/tasks/pile/pile_stackexchange.yaml'
2024-01-31T16:25:36,089   adding 'lm_eval/tasks/pile/pile_ubuntu-irc.yaml'
2024-01-31T16:25:36,090   adding 'lm_eval/tasks/pile/pile_uspto.yaml'
2024-01-31T16:25:36,091   adding 'lm_eval/tasks/pile/pile_wikipedia.yaml'
2024-01-31T16:25:36,092   adding 'lm_eval/tasks/pile/pile_youtubesubtitles.yaml'
2024-01-31T16:25:36,094   adding 'lm_eval/tasks/piqa/README.md'
2024-01-31T16:25:36,095   adding 'lm_eval/tasks/piqa/piqa.yaml'
2024-01-31T16:25:36,097   adding 'lm_eval/tasks/polemo2/README.md'
2024-01-31T16:25:36,098   adding 'lm_eval/tasks/polemo2/polemo2_in.yaml'
2024-01-31T16:25:36,099   adding 'lm_eval/tasks/polemo2/polemo2_out.yaml'
2024-01-31T16:25:36,101   adding 'lm_eval/tasks/prost/README.md'
2024-01-31T16:25:36,102   adding 'lm_eval/tasks/prost/corypaik_prost.yaml'
2024-01-31T16:25:36,104   adding 'lm_eval/tasks/pubmedqa/README.md'
2024-01-31T16:25:36,105   adding 'lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py'
2024-01-31T16:25:36,106   adding 'lm_eval/tasks/pubmedqa/pubmedqa.yaml'
2024-01-31T16:25:36,108   adding 'lm_eval/tasks/qa4mre/README.md'
2024-01-31T16:25:36,109   adding 'lm_eval/tasks/qa4mre/preprocess_qa4mre.py'
2024-01-31T16:25:36,110   adding 'lm_eval/tasks/qa4mre/qa4mre_2011.yaml'
2024-01-31T16:25:36,111   adding 'lm_eval/tasks/qa4mre/qa4mre_2012.yaml'
2024-01-31T16:25:36,112   adding 'lm_eval/tasks/qa4mre/qa4mre_2013.yaml'
2024-01-31T16:25:36,114   adding 'lm_eval/tasks/qasper/README.md'
2024-01-31T16:25:36,115   adding 'lm_eval/tasks/qasper/bool.yaml'
2024-01-31T16:25:36,117   adding 'lm_eval/tasks/qasper/freeform.yaml'
2024-01-31T16:25:36,118   adding 'lm_eval/tasks/qasper/metrics.py'
2024-01-31T16:25:36,119   adding 'lm_eval/tasks/qasper/utils.py'
2024-01-31T16:25:36,121   adding 'lm_eval/tasks/race/README.md'
2024-01-31T16:25:36,122   adding 'lm_eval/tasks/race/preprocess_race.py'
2024-01-31T16:25:36,123   adding 'lm_eval/tasks/race/race.yaml'
2024-01-31T16:25:36,125   adding 'lm_eval/tasks/realtoxicityprompts/metric.py'
2024-01-31T16:25:36,126   adding 'lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml'
2024-01-31T16:25:36,128   adding 'lm_eval/tasks/sciq/README.md'
2024-01-31T16:25:36,129   adding 'lm_eval/tasks/sciq/sciq.yaml'
2024-01-31T16:25:36,130   adding 'lm_eval/tasks/scrolls/README.md'
2024-01-31T16:25:36,131   adding 'lm_eval/tasks/scrolls/scrolls.yaml'
2024-01-31T16:25:36,133   adding 'lm_eval/tasks/scrolls/task.py'
2024-01-31T16:25:36,135   adding 'lm_eval/tasks/siqa/README.md'
2024-01-31T16:25:36,136   adding 'lm_eval/tasks/siqa/siqa.yaml'
2024-01-31T16:25:36,138   adding 'lm_eval/tasks/squadv2/README.md'
2024-01-31T16:25:36,140   adding 'lm_eval/tasks/squadv2/task.py'
2024-01-31T16:25:36,141   adding 'lm_eval/tasks/storycloze/README.md'
2024-01-31T16:25:36,142   adding 'lm_eval/tasks/storycloze/storycloze_2016.yaml'
2024-01-31T16:25:36,144   adding 'lm_eval/tasks/storycloze/storycloze_2018.yaml'
2024-01-31T16:25:36,145   adding 'lm_eval/tasks/super_glue/README.md'
2024-01-31T16:25:36,147   adding 'lm_eval/tasks/super_glue/boolq/default.yaml'
2024-01-31T16:25:36,148   adding 'lm_eval/tasks/super_glue/boolq/seq2seq.yaml'
2024-01-31T16:25:36,149   adding 'lm_eval/tasks/super_glue/boolq/t5-prompt.yaml'
2024-01-31T16:25:36,151   adding 'lm_eval/tasks/super_glue/cb/aggregate.py'
2024-01-31T16:25:36,152   adding 'lm_eval/tasks/super_glue/cb/default.yaml'
2024-01-31T16:25:36,153   adding 'lm_eval/tasks/super_glue/cb/t5-prompt.yaml'
2024-01-31T16:25:36,154   adding 'lm_eval/tasks/super_glue/cb/t5_utils.py'
2024-01-31T16:25:36,156   adding 'lm_eval/tasks/super_glue/copa/default.yaml'
2024-01-31T16:25:36,157   adding 'lm_eval/tasks/super_glue/copa/t5-prompt.yaml'
2024-01-31T16:25:36,158   adding 'lm_eval/tasks/super_glue/copa/utils.py'
2024-01-31T16:25:36,159   adding 'lm_eval/tasks/super_glue/multirc/default.yaml'
2024-01-31T16:25:36,161   adding 'lm_eval/tasks/super_glue/multirc/t5-prompt.yaml'
2024-01-31T16:25:36,162   adding 'lm_eval/tasks/super_glue/multirc/t5_utils.py'
2024-01-31T16:25:36,164   adding 'lm_eval/tasks/super_glue/record/default.yaml'
2024-01-31T16:25:36,165   adding 'lm_eval/tasks/super_glue/record/t5-prompt.yaml'
2024-01-31T16:25:36,166   adding 'lm_eval/tasks/super_glue/record/t5_utils.py'
2024-01-31T16:25:36,167   adding 'lm_eval/tasks/super_glue/record/util.py'
2024-01-31T16:25:36,169   adding 'lm_eval/tasks/super_glue/rte/default.yaml'
2024-01-31T16:25:36,170   adding 'lm_eval/tasks/super_glue/rte/t5-prompt.yaml'
2024-01-31T16:25:36,172   adding 'lm_eval/tasks/super_glue/wic/default.yaml'
2024-01-31T16:25:36,173   adding 'lm_eval/tasks/super_glue/wic/t5-prompt.yaml'
2024-01-31T16:25:36,175   adding 'lm_eval/tasks/super_glue/wsc/default.yaml'
2024-01-31T16:25:36,176   adding 'lm_eval/tasks/super_glue/wsc/preprocess_wsc.py'
2024-01-31T16:25:36,177   adding 'lm_eval/tasks/super_glue/wsc/t5-prompt.yaml'
2024-01-31T16:25:36,178   adding 'lm_eval/tasks/super_glue/wsc/t5_utils.py'
2024-01-31T16:25:36,180   adding 'lm_eval/tasks/swag/README.md'
2024-01-31T16:25:36,181   adding 'lm_eval/tasks/swag/swag.yaml'
2024-01-31T16:25:36,183   adding 'lm_eval/tasks/toxigen/README.md'
2024-01-31T16:25:36,184   adding 'lm_eval/tasks/toxigen/toxigen.yaml'
2024-01-31T16:25:36,186   adding 'lm_eval/tasks/toxigen/utils.py'
2024-01-31T16:25:36,187   adding 'lm_eval/tasks/translation/README.md'
2024-01-31T16:25:36,188   adding 'lm_eval/tasks/translation/iwslt2017_ar-en.yaml'
2024-01-31T16:25:36,190   adding 'lm_eval/tasks/translation/iwslt2017_en-ar.yaml'
2024-01-31T16:25:36,191   adding 'lm_eval/tasks/translation/utils.py'
2024-01-31T16:25:36,192   adding 'lm_eval/tasks/translation/wmt14_en-fr.yaml'
2024-01-31T16:25:36,193   adding 'lm_eval/tasks/translation/wmt14_fr-en.yaml'
2024-01-31T16:25:36,194   adding 'lm_eval/tasks/translation/wmt16_de-en.yaml'
2024-01-31T16:25:36,196   adding 'lm_eval/tasks/translation/wmt16_en-de.yaml'
2024-01-31T16:25:36,197   adding 'lm_eval/tasks/translation/wmt16_en-ro.yaml'
2024-01-31T16:25:36,198   adding 'lm_eval/tasks/translation/wmt16_ro-en.yaml'
2024-01-31T16:25:36,199   adding 'lm_eval/tasks/translation/wmt_common_yaml'
2024-01-31T16:25:36,201   adding 'lm_eval/tasks/triviaqa/README.md'
2024-01-31T16:25:36,202   adding 'lm_eval/tasks/triviaqa/default.yaml'
2024-01-31T16:25:36,203   adding 'lm_eval/tasks/truthfulqa/README.md'
2024-01-31T16:25:36,205   adding 'lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml'
2024-01-31T16:25:36,206   adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml'
2024-01-31T16:25:36,207   adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml'
2024-01-31T16:25:36,208   adding 'lm_eval/tasks/truthfulqa/utils.py'
2024-01-31T16:25:36,210   adding 'lm_eval/tasks/unscramble/README.md'
2024-01-31T16:25:36,211   adding 'lm_eval/tasks/unscramble/anagrams1.yaml'
2024-01-31T16:25:36,212   adding 'lm_eval/tasks/unscramble/anagrams2.yaml'
2024-01-31T16:25:36,213   adding 'lm_eval/tasks/unscramble/cycle_letters.yaml'
2024-01-31T16:25:36,214   adding 'lm_eval/tasks/unscramble/random_insertion.yaml'
2024-01-31T16:25:36,215   adding 'lm_eval/tasks/unscramble/reversed_words.yaml'
2024-01-31T16:25:36,217   adding 'lm_eval/tasks/webqs/README.md'
2024-01-31T16:25:36,218   adding 'lm_eval/tasks/webqs/utils.py'
2024-01-31T16:25:36,219   adding 'lm_eval/tasks/webqs/webqs.yaml'
2024-01-31T16:25:36,221   adding 'lm_eval/tasks/wikitext/README.md'
2024-01-31T16:25:36,222   adding 'lm_eval/tasks/wikitext/preprocess_wikitext.py'
2024-01-31T16:25:36,223   adding 'lm_eval/tasks/wikitext/wikitext.yaml'
2024-01-31T16:25:36,225   adding 'lm_eval/tasks/winogrande/README.md'
2024-01-31T16:25:36,226   adding 'lm_eval/tasks/winogrande/default.yaml'
2024-01-31T16:25:36,228   adding 'lm_eval/tasks/winogrande/preprocess_winogrande.py'
2024-01-31T16:25:36,229   adding 'lm_eval/tasks/wmt2016/README.md'
2024-01-31T16:25:36,231   adding 'lm_eval/tasks/wmt2016/metrics.py'
2024-01-31T16:25:36,232   adding 'lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml'
2024-01-31T16:25:36,234   adding 'lm_eval/tasks/wsc273/README.md'
2024-01-31T16:25:36,235   adding 'lm_eval/tasks/wsc273/default.yaml'
2024-01-31T16:25:36,236   adding 'lm_eval/tasks/wsc273/utils.py'
2024-01-31T16:25:36,238   adding 'lm_eval/tasks/xcopa/README.md'
2024-01-31T16:25:36,239   adding 'lm_eval/tasks/xcopa/default_et.yaml'
2024-01-31T16:25:36,240   adding 'lm_eval/tasks/xcopa/default_ht.yaml'
2024-01-31T16:25:36,241   adding 'lm_eval/tasks/xcopa/default_id.yaml'
2024-01-31T16:25:36,242   adding 'lm_eval/tasks/xcopa/default_it.yaml'
2024-01-31T16:25:36,243   adding 'lm_eval/tasks/xcopa/default_qu.yaml'
2024-01-31T16:25:36,245   adding 'lm_eval/tasks/xcopa/default_sw.yaml'
2024-01-31T16:25:36,246   adding 'lm_eval/tasks/xcopa/default_ta.yaml'
2024-01-31T16:25:36,247   adding 'lm_eval/tasks/xcopa/default_th.yaml'
2024-01-31T16:25:36,248   adding 'lm_eval/tasks/xcopa/default_tr.yaml'
2024-01-31T16:25:36,249   adding 'lm_eval/tasks/xcopa/default_vi.yaml'
2024-01-31T16:25:36,250   adding 'lm_eval/tasks/xcopa/default_zh.yaml'
2024-01-31T16:25:36,251   adding 'lm_eval/tasks/xcopa/utils.py'
2024-01-31T16:25:36,253   adding 'lm_eval/tasks/xnli/README.md'
2024-01-31T16:25:36,255   adding 'lm_eval/tasks/xnli/utils.py'
2024-01-31T16:25:36,256   adding 'lm_eval/tasks/xnli/xnli_ar.yaml'
2024-01-31T16:25:36,257   adding 'lm_eval/tasks/xnli/xnli_bg.yaml'
2024-01-31T16:25:36,258   adding 'lm_eval/tasks/xnli/xnli_common_yaml'
2024-01-31T16:25:36,259   adding 'lm_eval/tasks/xnli/xnli_de.yaml'
2024-01-31T16:25:36,260   adding 'lm_eval/tasks/xnli/xnli_el.yaml'
2024-01-31T16:25:36,262   adding 'lm_eval/tasks/xnli/xnli_en.yaml'
2024-01-31T16:25:36,263   adding 'lm_eval/tasks/xnli/xnli_es.yaml'
2024-01-31T16:25:36,264   adding 'lm_eval/tasks/xnli/xnli_fr.yaml'
2024-01-31T16:25:36,265   adding 'lm_eval/tasks/xnli/xnli_hi.yaml'
2024-01-31T16:25:36,266   adding 'lm_eval/tasks/xnli/xnli_ru.yaml'
2024-01-31T16:25:36,267   adding 'lm_eval/tasks/xnli/xnli_sw.yaml'
2024-01-31T16:25:36,268   adding 'lm_eval/tasks/xnli/xnli_th.yaml'
2024-01-31T16:25:36,270   adding 'lm_eval/tasks/xnli/xnli_tr.yaml'
2024-01-31T16:25:36,271   adding 'lm_eval/tasks/xnli/xnli_ur.yaml'
2024-01-31T16:25:36,272   adding 'lm_eval/tasks/xnli/xnli_vi.yaml'
2024-01-31T16:25:36,273   adding 'lm_eval/tasks/xnli/xnli_zh.yaml'
2024-01-31T16:25:36,274   adding 'lm_eval/tasks/xstorycloze/README.md'
2024-01-31T16:25:36,275   adding 'lm_eval/tasks/xstorycloze/default_ar.yaml'
2024-01-31T16:25:36,276   adding 'lm_eval/tasks/xstorycloze/default_en.yaml'
2024-01-31T16:25:36,277   adding 'lm_eval/tasks/xstorycloze/default_es.yaml'
2024-01-31T16:25:36,278   adding 'lm_eval/tasks/xstorycloze/default_eu.yaml'
2024-01-31T16:25:36,279   adding 'lm_eval/tasks/xstorycloze/default_hi.yaml'
2024-01-31T16:25:36,280   adding 'lm_eval/tasks/xstorycloze/default_id.yaml'
2024-01-31T16:25:36,281   adding 'lm_eval/tasks/xstorycloze/default_my.yaml'
2024-01-31T16:25:36,282   adding 'lm_eval/tasks/xstorycloze/default_ru.yaml'
2024-01-31T16:25:36,284   adding 'lm_eval/tasks/xstorycloze/default_sw.yaml'
2024-01-31T16:25:36,285   adding 'lm_eval/tasks/xstorycloze/default_te.yaml'
2024-01-31T16:25:36,286   adding 'lm_eval/tasks/xstorycloze/default_zh.yaml'
2024-01-31T16:25:36,287   adding 'lm_eval/tasks/xwinograd/README.md'
2024-01-31T16:25:36,289   adding 'lm_eval/tasks/xwinograd/utils.py'
2024-01-31T16:25:36,290   adding 'lm_eval/tasks/xwinograd/xwinograd_common_yaml'
2024-01-31T16:25:36,291   adding 'lm_eval/tasks/xwinograd/xwinograd_en.yaml'
2024-01-31T16:25:36,292   adding 'lm_eval/tasks/xwinograd/xwinograd_fr.yaml'
2024-01-31T16:25:36,293   adding 'lm_eval/tasks/xwinograd/xwinograd_jp.yaml'
2024-01-31T16:25:36,294   adding 'lm_eval/tasks/xwinograd/xwinograd_pt.yaml'
2024-01-31T16:25:36,295   adding 'lm_eval/tasks/xwinograd/xwinograd_ru.yaml'
2024-01-31T16:25:36,296   adding 'lm_eval/tasks/xwinograd/xwinograd_zh.yaml'
2024-01-31T16:25:36,298   adding 'lm_eval-0.4.1.dist-info/LICENSE.md'
2024-01-31T16:25:36,302   adding 'lm_eval-0.4.1.dist-info/METADATA'
2024-01-31T16:25:36,303   adding 'lm_eval-0.4.1.dist-info/WHEEL'
2024-01-31T16:25:36,304   adding 'lm_eval-0.4.1.dist-info/entry_points.txt'
2024-01-31T16:25:36,305   adding 'lm_eval-0.4.1.dist-info/top_level.txt'
2024-01-31T16:25:36,336   adding 'lm_eval-0.4.1.dist-info/RECORD'
2024-01-31T16:25:36,368   removing build/bdist.linux-armv7l/wheel
2024-01-31T16:25:36,852   Building wheel for lm-eval (pyproject.toml): finished with status 'done'
2024-01-31T16:25:36,876   Created wheel for lm-eval: filename=lm_eval-0.4.1-py3-none-any.whl size=1068385 sha256=796615edf52f5673187a884a9d77c7d755fe95ca2d16fe77a1f042730eb957b3
2024-01-31T16:25:36,877   Stored in directory: /tmp/pip-ephem-wheel-cache-ctj9ybzs/wheels/4b/d9/61/e078678c56f6b0f14f82fba8954d0f6e3f64580ea95741e83f
2024-01-31T16:25:36,945 Successfully built lm-eval
2024-01-31T16:25:36,980 Removed build tracker: '/tmp/pip-build-tracker-eq5cqtfn'