2023-09-06T15:57:12,115 Created temporary directory: /tmp/pip-build-tracker-4h7t43l0 2023-09-06T15:57:12,119 Initialized build tracking at /tmp/pip-build-tracker-4h7t43l0 2023-09-06T15:57:12,119 Created build tracker: /tmp/pip-build-tracker-4h7t43l0 2023-09-06T15:57:12,119 Entered build tracker: /tmp/pip-build-tracker-4h7t43l0 2023-09-06T15:57:12,121 Created temporary directory: /tmp/pip-wheel-czpwlzl0 2023-09-06T15:57:12,131 Created temporary directory: /tmp/pip-ephem-wheel-cache-2w7bmwd7 2023-09-06T15:57:12,192 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2023-09-06T15:57:12,200 2 location(s) to search for versions of colossalai: 2023-09-06T15:57:12,200 * https://pypi.org/simple/colossalai/ 2023-09-06T15:57:12,200 * https://www.piwheels.org/simple/colossalai/ 2023-09-06T15:57:12,201 Fetching project page and analyzing links: https://pypi.org/simple/colossalai/ 2023-09-06T15:57:12,202 Getting page https://pypi.org/simple/colossalai/ 2023-09-06T15:57:12,211 Found index url https://pypi.org/simple/ 2023-09-06T15:57:12,400 Fetched page https://pypi.org/simple/colossalai/ as application/vnd.pypi.simple.v1+json 2023-09-06T15:57:12,408 Found link https://files.pythonhosted.org/packages/9c/c7/427dd2ed3769c6beb05cefdc18f8fad634673ed1472d5409afbdb768c7c3/colossalai-0.2.0.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.0 2023-09-06T15:57:12,410 Found link https://files.pythonhosted.org/packages/b1/70/cc6165ba3ce239c4b0c53c8ea6cabbfb31aa4b66dc580e85ff16d787384d/colossalai-0.2.1.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.1 2023-09-06T15:57:12,411 Found link https://files.pythonhosted.org/packages/87/d0/d82b49b722315654f6773eb458b929dba9949044a6b7cd087027c2391169/colossalai-0.2.2.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.2 2023-09-06T15:57:12,412 Found link https://files.pythonhosted.org/packages/c9/bb/9ec93ed622207b589fe4b1eddad9da74174eedc40ac306bff77ba823db92/colossalai-0.2.3.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.3 2023-09-06T15:57:12,414 Found link https://files.pythonhosted.org/packages/2c/26/4c38cbad383dfae44b9ececdea8f71d1be32c0ce80aadf883f915dd4d2fb/colossalai-0.2.4.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.4 2023-09-06T15:57:12,415 Found link https://files.pythonhosted.org/packages/15/05/38488d4a76c1c010da8d07e07ba14b15191d0e7398057a683f34e3c54dfe/colossalai-0.2.5.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.5 2023-09-06T15:57:12,417 Found link https://files.pythonhosted.org/packages/5a/a3/edf03a61814ca761d3133fae4bad80473c0f04693ef4af083787e71b6ac9/colossalai-0.2.6.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.6 2023-09-06T15:57:12,418 Found link https://files.pythonhosted.org/packages/25/7c/99446b8f26ba3a8706553868aa230dd9e9e153f024961967c6d916ddc943/colossalai-0.2.7.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.7 2023-09-06T15:57:12,420 Found link https://files.pythonhosted.org/packages/8a/35/c33a150a97c33973318108778a559a9d550f4dc89757d40f23f3260e5af8/colossalai-0.2.8.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.8 2023-09-06T15:57:12,421 Found link https://files.pythonhosted.org/packages/ef/89/290c2fc3752a3ec9008f83cf6e8cae9c856026559c8a64683b00b1501a4f/colossalai-0.3.0.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.3.0 2023-09-06T15:57:12,422 Found link https://files.pythonhosted.org/packages/a8/85/2d687fa9176a8ee5a77b00046ece47e9742c99a98e73b6afc7aa28f118c7/colossalai-0.3.1.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.3.1 2023-09-06T15:57:12,424 Found link https://files.pythonhosted.org/packages/dd/57/451c37bf3d47fb6946ab4760910e42ff44408613abd4fceeaf9e127e5dc1/colossalai-0.3.2.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.3.2 2023-09-06T15:57:12,425 Fetching project page and analyzing links: https://www.piwheels.org/simple/colossalai/ 2023-09-06T15:57:12,425 Getting page https://www.piwheels.org/simple/colossalai/ 2023-09-06T15:57:12,428 Found index url https://www.piwheels.org/simple/ 2023-09-06T15:57:12,665 Fetched page https://www.piwheels.org/simple/colossalai/ as text/html 2023-09-06T15:57:12,675 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.3.1-py3-none-any.whl#sha256=7496490930a13a297e1773ec3fac4ac473eafb09bb98d3d882b94c09bca15533 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,676 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.3.0-py3-none-any.whl#sha256=33c149e9cd560d2c3e37c8d19bbd906fccccd7ba9703ef350d081e24b52ba6db (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,676 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.8-py3-none-any.whl#sha256=4692d233a9b6b43dc1fb53ff2aa8de5a78ad7abf07bb21c9e46ddd48f330f666 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,677 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.7-py3-none-any.whl#sha256=d8c17d923035a789da65ce205a0daea196df83cec28f43390376dfca88775424 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,677 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.6-py3-none-any.whl#sha256=2f5357caa65bb726a2aadadbd876c6f53f87d5984bb016ccdfb411f9ccf13e2e (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,678 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.5-py3-none-any.whl#sha256=645c27a6955e81220cbf83dbc4eec810b1b271bdbd2a695bd3f1a179a2a348ff (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,678 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.4-py3-none-any.whl#sha256=4c6554305658e978bfa9b199fd59b284ba6fb60cf9b17002bed4982c13f17c1f (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,679 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.3-py3-none-any.whl#sha256=e5c1d7c77bd0eb9c0b3cea5c33050a759bb29fcd128ef0f6459bec9e69f4969c (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,679 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.2-py3-none-any.whl#sha256=272ef339dfcdc089fc4e150a9ed2377e0998b1b3e5061b8760add23228609122 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,680 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.1-py3-none-any.whl#sha256=fba9ebaec6b5f43d7e2424a858cae18f765cf95d36956496e4476f23f36d97b0 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,680 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.0-py3-none-any.whl#sha256=707a78c4c8d4b96171adc704e95bbdc387fc1dee4f612a8500f0a562c4454dc7 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-09-06T15:57:12,681 Skipping link: not a file: https://www.piwheels.org/simple/colossalai/ 2023-09-06T15:57:12,681 Skipping link: not a file: https://pypi.org/simple/colossalai/ 2023-09-06T15:57:12,715 Given no hashes to check 1 links for project 'colossalai': discarding no candidates 2023-09-06T15:57:12,744 Collecting colossalai==0.3.2 2023-09-06T15:57:12,749 Created temporary directory: /tmp/pip-unpack-4n6ji_3t 2023-09-06T15:57:12,934 Downloading colossalai-0.3.2.tar.gz (917 kB) 2023-09-06T15:57:17,003 Added colossalai==0.3.2 from https://files.pythonhosted.org/packages/dd/57/451c37bf3d47fb6946ab4760910e42ff44408613abd4fceeaf9e127e5dc1/colossalai-0.3.2.tar.gz to build tracker '/tmp/pip-build-tracker-4h7t43l0' 2023-09-06T15:57:17,007 Running setup.py (path:/tmp/pip-wheel-czpwlzl0/colossalai_ced65a84bfa342caa16701a8159994de/setup.py) egg_info for package colossalai 2023-09-06T15:57:17,008 Created temporary directory: /tmp/pip-pip-egg-info-e57_wz96 2023-09-06T15:57:17,009 Preparing metadata (setup.py): started 2023-09-06T15:57:17,011 Running command python setup.py egg_info 2023-09-06T15:57:19,749 running egg_info 2023-09-06T15:57:19,751 creating /tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info 2023-09-06T15:57:19,807 writing /tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/PKG-INFO 2023-09-06T15:57:19,813 writing dependency_links to /tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/dependency_links.txt 2023-09-06T15:57:19,816 writing entry points to /tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/entry_points.txt 2023-09-06T15:57:19,819 writing requirements to /tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/requires.txt 2023-09-06T15:57:19,822 writing top-level names to /tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/top_level.txt 2023-09-06T15:57:19,824 writing manifest file '/tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/SOURCES.txt' 2023-09-06T15:57:20,238 reading manifest file '/tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/SOURCES.txt' 2023-09-06T15:57:20,242 reading manifest template 'MANIFEST.in' 2023-09-06T15:57:22,607 warning: no files found matching '*.tr' under directory 'colossalai' 2023-09-06T15:57:23,805 warning: no files found matching '*.cc' under directory 'colossalai' 2023-09-06T15:57:24,410 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-09-06T15:57:24,428 adding license file 'LICENSE' 2023-09-06T15:57:24,505 writing manifest file '/tmp/pip-pip-egg-info-e57_wz96/colossalai.egg-info/SOURCES.txt' 2023-09-06T15:57:24,673 Preparing metadata (setup.py): finished with status 'done' 2023-09-06T15:57:24,690 Source in /tmp/pip-wheel-czpwlzl0/colossalai_ced65a84bfa342caa16701a8159994de has version 0.3.2, which satisfies requirement colossalai==0.3.2 from https://files.pythonhosted.org/packages/dd/57/451c37bf3d47fb6946ab4760910e42ff44408613abd4fceeaf9e127e5dc1/colossalai-0.3.2.tar.gz 2023-09-06T15:57:24,692 Removed colossalai==0.3.2 from https://files.pythonhosted.org/packages/dd/57/451c37bf3d47fb6946ab4760910e42ff44408613abd4fceeaf9e127e5dc1/colossalai-0.3.2.tar.gz from build tracker '/tmp/pip-build-tracker-4h7t43l0' 2023-09-06T15:57:24,712 Created temporary directory: /tmp/pip-unpack-oep6_gws 2023-09-06T15:57:24,714 Created temporary directory: /tmp/pip-unpack-p74ghyjr 2023-09-06T15:57:24,747 Building wheels for collected packages: colossalai 2023-09-06T15:57:24,756 Created temporary directory: /tmp/pip-wheel-gra_qedi 2023-09-06T15:57:24,757 Building wheel for colossalai (setup.py): started 2023-09-06T15:57:24,759 Destination directory: /tmp/pip-wheel-gra_qedi 2023-09-06T15:57:24,759 Running command python setup.py bdist_wheel 2023-09-06T15:57:27,250 running bdist_wheel 2023-09-06T15:57:27,415 running build 2023-09-06T15:57:27,415 running build_py 2023-09-06T15:57:27,478 creating build 2023-09-06T15:57:27,479 creating build/lib 2023-09-06T15:57:27,480 creating build/lib/colossalai 2023-09-06T15:57:27,482 copying colossalai/version.py -> build/lib/colossalai 2023-09-06T15:57:27,486 copying colossalai/core.py -> build/lib/colossalai 2023-09-06T15:57:27,489 copying colossalai/__init__.py -> build/lib/colossalai 2023-09-06T15:57:27,492 copying colossalai/global_variables.py -> build/lib/colossalai 2023-09-06T15:57:27,496 copying colossalai/initialize.py -> build/lib/colossalai 2023-09-06T15:57:27,501 copying colossalai/constants.py -> build/lib/colossalai 2023-09-06T15:57:27,505 creating build/lib/colossalai/kernel 2023-09-06T15:57:27,507 copying colossalai/kernel/__init__.py -> build/lib/colossalai/kernel 2023-09-06T15:57:27,512 creating build/lib/colossalai/context 2023-09-06T15:57:27,513 copying colossalai/context/moe_context.py -> build/lib/colossalai/context 2023-09-06T15:57:27,518 copying colossalai/context/parallel_context.py -> build/lib/colossalai/context 2023-09-06T15:57:27,522 copying colossalai/context/__init__.py -> build/lib/colossalai/context 2023-09-06T15:57:27,526 copying colossalai/context/parallel_mode.py -> build/lib/colossalai/context 2023-09-06T15:57:27,529 copying colossalai/context/config.py -> build/lib/colossalai/context 2023-09-06T15:57:27,532 copying colossalai/context/singleton_meta.py -> build/lib/colossalai/context 2023-09-06T15:57:27,536 creating build/lib/colossalai/amp 2023-09-06T15:57:27,538 copying colossalai/amp/__init__.py -> build/lib/colossalai/amp 2023-09-06T15:57:27,542 copying colossalai/amp/amp_type.py -> build/lib/colossalai/amp 2023-09-06T15:57:27,546 creating build/lib/colossalai/auto_parallel 2023-09-06T15:57:27,548 copying colossalai/auto_parallel/__init__.py -> build/lib/colossalai/auto_parallel 2023-09-06T15:57:27,551 creating build/lib/colossalai/_C 2023-09-06T15:57:27,553 copying colossalai/_C/__init__.py -> build/lib/colossalai/_C 2023-09-06T15:57:27,557 creating build/lib/colossalai/checkpoint_io 2023-09-06T15:57:27,559 copying colossalai/checkpoint_io/__init__.py -> build/lib/colossalai/checkpoint_io 2023-09-06T15:57:27,562 copying colossalai/checkpoint_io/general_checkpoint_io.py -> build/lib/colossalai/checkpoint_io 2023-09-06T15:57:27,567 copying colossalai/checkpoint_io/checkpoint_io_base.py -> build/lib/colossalai/checkpoint_io 2023-09-06T15:57:27,571 copying colossalai/checkpoint_io/hybrid_parallel_checkpoint_io.py -> build/lib/colossalai/checkpoint_io 2023-09-06T15:57:27,576 copying colossalai/checkpoint_io/index_file.py -> build/lib/colossalai/checkpoint_io 2023-09-06T15:57:27,580 copying colossalai/checkpoint_io/utils.py -> build/lib/colossalai/checkpoint_io 2023-09-06T15:57:27,586 creating build/lib/colossalai/cli 2023-09-06T15:57:27,588 copying colossalai/cli/cli.py -> build/lib/colossalai/cli 2023-09-06T15:57:27,591 copying colossalai/cli/__init__.py -> build/lib/colossalai/cli 2023-09-06T15:57:27,596 creating build/lib/colossalai/fx 2023-09-06T15:57:27,597 copying colossalai/fx/_meta_regist_12.py -> build/lib/colossalai/fx 2023-09-06T15:57:27,602 copying colossalai/fx/__init__.py -> build/lib/colossalai/fx 2023-09-06T15:57:27,606 copying colossalai/fx/_meta_regist_13.py -> build/lib/colossalai/fx 2023-09-06T15:57:27,609 copying colossalai/fx/graph_module.py -> build/lib/colossalai/fx 2023-09-06T15:57:27,612 copying colossalai/fx/proxy.py -> build/lib/colossalai/fx 2023-09-06T15:57:27,616 copying colossalai/fx/_compatibility.py -> build/lib/colossalai/fx 2023-09-06T15:57:27,621 creating build/lib/colossalai/pipeline 2023-09-06T15:57:27,622 copying colossalai/pipeline/pipelinable.py -> build/lib/colossalai/pipeline 2023-09-06T15:57:27,627 copying colossalai/pipeline/stage_manager.py -> build/lib/colossalai/pipeline 2023-09-06T15:57:27,631 copying colossalai/pipeline/__init__.py -> build/lib/colossalai/pipeline 2023-09-06T15:57:27,634 copying colossalai/pipeline/layer_spec.py -> build/lib/colossalai/pipeline 2023-09-06T15:57:27,638 copying colossalai/pipeline/pipeline_process_group.py -> build/lib/colossalai/pipeline 2023-09-06T15:57:27,642 copying colossalai/pipeline/p2p.py -> build/lib/colossalai/pipeline 2023-09-06T15:57:27,646 copying colossalai/pipeline/utils.py -> build/lib/colossalai/pipeline 2023-09-06T15:57:27,652 creating build/lib/colossalai/lazy 2023-09-06T15:57:27,653 copying colossalai/lazy/__init__.py -> build/lib/colossalai/lazy 2023-09-06T15:57:27,657 copying colossalai/lazy/lazy_init.py -> build/lib/colossalai/lazy 2023-09-06T15:57:27,664 creating build/lib/colossalai/tensor 2023-09-06T15:57:27,666 copying colossalai/tensor/shape_consistency.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,673 copying colossalai/tensor/tensor_spec.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,677 copying colossalai/tensor/sharding_spec.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,682 copying colossalai/tensor/dist_spec_mgr.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,687 copying colossalai/tensor/op_wrapper.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,690 copying colossalai/tensor/__init__.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,694 copying colossalai/tensor/const.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,698 copying colossalai/tensor/process_group.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,703 copying colossalai/tensor/colo_tensor.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,707 copying colossalai/tensor/compute_spec.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,710 copying colossalai/tensor/colo_parameter.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,714 copying colossalai/tensor/param_op_hook.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,719 copying colossalai/tensor/distspec.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,723 copying colossalai/tensor/utils.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,728 copying colossalai/tensor/comm_spec.py -> build/lib/colossalai/tensor 2023-09-06T15:57:27,734 creating build/lib/colossalai/_analyzer 2023-09-06T15:57:27,736 copying colossalai/_analyzer/__init__.py -> build/lib/colossalai/_analyzer 2023-09-06T15:57:27,739 copying colossalai/_analyzer/envs.py -> build/lib/colossalai/_analyzer 2023-09-06T15:57:27,743 creating build/lib/colossalai/interface 2023-09-06T15:57:27,745 copying colossalai/interface/model.py -> build/lib/colossalai/interface 2023-09-06T15:57:27,749 copying colossalai/interface/optimizer.py -> build/lib/colossalai/interface 2023-09-06T15:57:27,753 copying colossalai/interface/__init__.py -> build/lib/colossalai/interface 2023-09-06T15:57:27,758 creating build/lib/colossalai/booster 2023-09-06T15:57:27,760 copying colossalai/booster/booster.py -> build/lib/colossalai/booster 2023-09-06T15:57:27,764 copying colossalai/booster/__init__.py -> build/lib/colossalai/booster 2023-09-06T15:57:27,768 copying colossalai/booster/accelerator.py -> build/lib/colossalai/booster 2023-09-06T15:57:27,774 creating build/lib/colossalai/cluster 2023-09-06T15:57:27,775 copying colossalai/cluster/device_mesh_manager.py -> build/lib/colossalai/cluster 2023-09-06T15:57:27,779 copying colossalai/cluster/__init__.py -> build/lib/colossalai/cluster 2023-09-06T15:57:27,783 copying colossalai/cluster/dist_coordinator.py -> build/lib/colossalai/cluster 2023-09-06T15:57:27,787 copying colossalai/cluster/process_group_manager.py -> build/lib/colossalai/cluster 2023-09-06T15:57:27,790 copying colossalai/cluster/process_group_mesh.py -> build/lib/colossalai/cluster 2023-09-06T15:57:27,795 creating build/lib/colossalai/device 2023-09-06T15:57:27,797 copying colossalai/device/calc_pipeline_strategy.py -> build/lib/colossalai/device 2023-09-06T15:57:27,801 copying colossalai/device/alpha_beta_profiler.py -> build/lib/colossalai/device 2023-09-06T15:57:27,805 copying colossalai/device/__init__.py -> build/lib/colossalai/device 2023-09-06T15:57:27,808 copying colossalai/device/device_mesh.py -> build/lib/colossalai/device 2023-09-06T15:57:27,814 creating build/lib/colossalai/legacy 2023-09-06T15:57:27,816 copying colossalai/legacy/__init__.py -> build/lib/colossalai/legacy 2023-09-06T15:57:27,820 creating build/lib/colossalai/zero 2023-09-06T15:57:27,822 copying colossalai/zero/wrapper.py -> build/lib/colossalai/zero 2023-09-06T15:57:27,826 copying colossalai/zero/__init__.py -> build/lib/colossalai/zero 2023-09-06T15:57:27,831 creating build/lib/colossalai/testing 2023-09-06T15:57:27,833 copying colossalai/testing/pytest_wrapper.py -> build/lib/colossalai/testing 2023-09-06T15:57:27,836 copying colossalai/testing/random.py -> build/lib/colossalai/testing 2023-09-06T15:57:27,839 copying colossalai/testing/__init__.py -> build/lib/colossalai/testing 2023-09-06T15:57:27,843 copying colossalai/testing/comparison.py -> build/lib/colossalai/testing 2023-09-06T15:57:27,847 copying colossalai/testing/utils.py -> build/lib/colossalai/testing 2023-09-06T15:57:27,852 creating build/lib/colossalai/nn 2023-09-06T15:57:27,854 copying colossalai/nn/init.py -> build/lib/colossalai/nn 2023-09-06T15:57:27,858 copying colossalai/nn/__init__.py -> build/lib/colossalai/nn 2023-09-06T15:57:27,863 creating build/lib/colossalai/logging 2023-09-06T15:57:27,864 copying colossalai/logging/__init__.py -> build/lib/colossalai/logging 2023-09-06T15:57:27,868 copying colossalai/logging/logger.py -> build/lib/colossalai/logging 2023-09-06T15:57:27,873 creating build/lib/colossalai/shardformer 2023-09-06T15:57:27,875 copying colossalai/shardformer/__init__.py -> build/lib/colossalai/shardformer 2023-09-06T15:57:27,879 copying colossalai/shardformer/_utils.py -> build/lib/colossalai/shardformer 2023-09-06T15:57:27,883 creating build/lib/colossalai/communication 2023-09-06T15:57:27,885 copying colossalai/communication/p2p_v2.py -> build/lib/colossalai/communication 2023-09-06T15:57:27,889 copying colossalai/communication/__init__.py -> build/lib/colossalai/communication 2023-09-06T15:57:27,893 copying colossalai/communication/collective.py -> build/lib/colossalai/communication 2023-09-06T15:57:27,897 copying colossalai/communication/ring.py -> build/lib/colossalai/communication 2023-09-06T15:57:27,900 copying colossalai/communication/p2p.py -> build/lib/colossalai/communication 2023-09-06T15:57:27,905 copying colossalai/communication/utils.py -> build/lib/colossalai/communication 2023-09-06T15:57:27,911 creating build/lib/colossalai/utils 2023-09-06T15:57:27,912 copying colossalai/utils/moe.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,916 copying colossalai/utils/memory.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,920 copying colossalai/utils/__init__.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,924 copying colossalai/utils/cuda.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,927 copying colossalai/utils/common.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,932 copying colossalai/utils/checkpointing.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,937 copying colossalai/utils/activation_checkpoint.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,941 copying colossalai/utils/timer.py -> build/lib/colossalai/utils 2023-09-06T15:57:27,947 creating build/lib/colossalai/kernel/cuda_native 2023-09-06T15:57:27,949 copying colossalai/kernel/cuda_native/layer_norm.py -> build/lib/colossalai/kernel/cuda_native 2023-09-06T15:57:27,953 copying colossalai/kernel/cuda_native/__init__.py -> build/lib/colossalai/kernel/cuda_native 2023-09-06T15:57:27,956 copying colossalai/kernel/cuda_native/multihead_attention.py -> build/lib/colossalai/kernel/cuda_native 2023-09-06T15:57:27,961 copying colossalai/kernel/cuda_native/scaled_softmax.py -> build/lib/colossalai/kernel/cuda_native 2023-09-06T15:57:27,967 creating build/lib/colossalai/kernel/jit 2023-09-06T15:57:27,968 copying colossalai/kernel/jit/__init__.py -> build/lib/colossalai/kernel/jit 2023-09-06T15:57:27,972 copying colossalai/kernel/jit/bias_dropout_add.py -> build/lib/colossalai/kernel/jit 2023-09-06T15:57:27,976 copying colossalai/kernel/jit/option.py -> build/lib/colossalai/kernel/jit 2023-09-06T15:57:27,979 copying colossalai/kernel/jit/bias_gelu.py -> build/lib/colossalai/kernel/jit 2023-09-06T15:57:27,985 creating build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:27,987 copying colossalai/kernel/op_builder/moe.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:27,991 copying colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:27,994 copying colossalai/kernel/op_builder/fused_optim.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:27,998 copying colossalai/kernel/op_builder/cpu_adam.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:28,002 copying colossalai/kernel/op_builder/__init__.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:28,005 copying colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:28,009 copying colossalai/kernel/op_builder/layernorm.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:28,013 copying colossalai/kernel/op_builder/multi_head_attn.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:28,016 copying colossalai/kernel/op_builder/utils.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:28,021 copying colossalai/kernel/op_builder/builder.py -> build/lib/colossalai/kernel/op_builder 2023-09-06T15:57:28,027 creating build/lib/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:28,029 copying colossalai/kernel/cuda_native/mha/__init__.py -> build/lib/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:28,032 copying colossalai/kernel/cuda_native/mha/mem_eff_attn.py -> build/lib/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:28,036 copying colossalai/kernel/cuda_native/mha/mha.py -> build/lib/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:28,040 copying colossalai/kernel/cuda_native/mha/utils.py -> build/lib/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:28,044 copying colossalai/kernel/cuda_native/mha/flash_attn_2.py -> build/lib/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:28,049 creating build/lib/colossalai/context/random 2023-09-06T15:57:28,050 copying colossalai/context/random/_helper.py -> build/lib/colossalai/context/random 2023-09-06T15:57:28,054 copying colossalai/context/random/__init__.py -> build/lib/colossalai/context/random 2023-09-06T15:57:28,058 copying colossalai/context/random/seed_manager.py -> build/lib/colossalai/context/random 2023-09-06T15:57:28,063 creating build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,065 copying colossalai/context/process_group_initializer/initializer_2d.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,070 copying colossalai/context/process_group_initializer/initializer_tensor.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,073 copying colossalai/context/process_group_initializer/initializer_data.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,077 copying colossalai/context/process_group_initializer/__init__.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,080 copying colossalai/context/process_group_initializer/process_group_initializer.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,083 copying colossalai/context/process_group_initializer/initializer_3d.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,088 copying colossalai/context/process_group_initializer/initializer_sequence.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,092 copying colossalai/context/process_group_initializer/initializer_model.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,095 copying colossalai/context/process_group_initializer/initializer_1d.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,098 copying colossalai/context/process_group_initializer/initializer_pipeline.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,102 copying colossalai/context/process_group_initializer/initializer_2p5d.py -> build/lib/colossalai/context/process_group_initializer 2023-09-06T15:57:28,107 creating build/lib/colossalai/amp/torch_amp 2023-09-06T15:57:28,109 copying colossalai/amp/torch_amp/torch_amp.py -> build/lib/colossalai/amp/torch_amp 2023-09-06T15:57:28,113 copying colossalai/amp/torch_amp/__init__.py -> build/lib/colossalai/amp/torch_amp 2023-09-06T15:57:28,116 copying colossalai/amp/torch_amp/_grad_scaler.py -> build/lib/colossalai/amp/torch_amp 2023-09-06T15:57:28,122 creating build/lib/colossalai/amp/apex_amp 2023-09-06T15:57:28,124 copying colossalai/amp/apex_amp/__init__.py -> build/lib/colossalai/amp/apex_amp 2023-09-06T15:57:28,127 copying colossalai/amp/apex_amp/apex_amp.py -> build/lib/colossalai/amp/apex_amp 2023-09-06T15:57:28,132 creating build/lib/colossalai/amp/naive_amp 2023-09-06T15:57:28,134 copying colossalai/amp/naive_amp/mixed_precision_optimizer.py -> build/lib/colossalai/amp/naive_amp 2023-09-06T15:57:28,138 copying colossalai/amp/naive_amp/__init__.py -> build/lib/colossalai/amp/naive_amp 2023-09-06T15:57:28,142 copying colossalai/amp/naive_amp/naive_amp.py -> build/lib/colossalai/amp/naive_amp 2023-09-06T15:57:28,145 copying colossalai/amp/naive_amp/_fp16_optimizer.py -> build/lib/colossalai/amp/naive_amp 2023-09-06T15:57:28,151 copying colossalai/amp/naive_amp/_utils.py -> build/lib/colossalai/amp/naive_amp 2023-09-06T15:57:28,156 creating build/lib/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:28,158 copying colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:28,162 copying colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:28,165 copying colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:28,169 copying colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:28,174 creating build/lib/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:28,176 copying colossalai/amp/naive_amp/mixed_precision_mixin/__init__.py -> build/lib/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:28,179 copying colossalai/amp/naive_amp/mixed_precision_mixin/bf16.py -> build/lib/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:28,183 copying colossalai/amp/naive_amp/mixed_precision_mixin/base.py -> build/lib/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:28,187 copying colossalai/amp/naive_amp/mixed_precision_mixin/fp16.py -> build/lib/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:28,191 creating build/lib/colossalai/auto_parallel/pipeline_shard 2023-09-06T15:57:28,193 copying colossalai/auto_parallel/pipeline_shard/__init__.py -> build/lib/colossalai/auto_parallel/pipeline_shard 2023-09-06T15:57:28,198 creating build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,200 copying colossalai/auto_parallel/offload/amp_optimizer.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,204 copying colossalai/auto_parallel/offload/util.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,208 copying colossalai/auto_parallel/offload/solver.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,213 copying colossalai/auto_parallel/offload/__init__.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,216 copying colossalai/auto_parallel/offload/region_manager.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,221 copying colossalai/auto_parallel/offload/mem_optimize.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,224 copying colossalai/auto_parallel/offload/region.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,229 copying colossalai/auto_parallel/offload/runtime.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,233 copying colossalai/auto_parallel/offload/base_offload_module.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,237 copying colossalai/auto_parallel/offload/training_simulator.py -> build/lib/colossalai/auto_parallel/offload 2023-09-06T15:57:28,243 creating build/lib/colossalai/auto_parallel/passes 2023-09-06T15:57:28,244 copying colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-09-06T15:57:28,249 copying colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-09-06T15:57:28,254 copying colossalai/auto_parallel/passes/meta_info_prop.py -> build/lib/colossalai/auto_parallel/passes 2023-09-06T15:57:28,258 copying colossalai/auto_parallel/passes/__init__.py -> build/lib/colossalai/auto_parallel/passes 2023-09-06T15:57:28,261 copying colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-09-06T15:57:28,265 copying colossalai/auto_parallel/passes/constants.py -> build/lib/colossalai/auto_parallel/passes 2023-09-06T15:57:28,270 creating build/lib/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:28,272 copying colossalai/auto_parallel/tensor_shard/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:28,275 copying colossalai/auto_parallel/tensor_shard/options.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:28,278 copying colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:28,283 copying colossalai/auto_parallel/tensor_shard/initialize.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:28,287 copying colossalai/auto_parallel/tensor_shard/constants.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:28,292 creating build/lib/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:28,294 copying colossalai/auto_parallel/checkpoint/__init__.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:28,298 copying colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:28,303 copying colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:28,306 copying colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:28,309 copying colossalai/auto_parallel/checkpoint/operation.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:28,313 copying colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:28,319 creating build/lib/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:28,321 copying colossalai/auto_parallel/meta_profiler/registry.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:28,324 copying colossalai/auto_parallel/meta_profiler/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:28,328 copying colossalai/auto_parallel/meta_profiler/shard_metainfo.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:28,331 copying colossalai/auto_parallel/meta_profiler/constants.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:28,339 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,341 copying colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,346 copying colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,350 copying colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,355 copying colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,359 copying colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,362 copying colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,367 copying colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,371 copying colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,374 copying colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,379 copying colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,382 copying colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,386 copying colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,389 copying colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,393 copying colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,397 copying colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,401 copying colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,404 copying colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,408 copying colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,411 copying colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,415 copying colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,418 copying colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,422 copying colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,425 copying colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,429 copying colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,432 copying colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,437 copying colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,440 copying colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:28,445 creating build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:28,446 copying colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:28,452 copying colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:28,455 copying colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:28,459 copying colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:28,463 copying colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:28,469 creating build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:28,470 copying colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:28,475 copying colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:28,478 copying colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:28,482 copying colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:28,486 copying colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:28,490 copying colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:28,497 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,498 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,502 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,507 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,511 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,514 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,518 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,521 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,525 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,529 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,535 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,539 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,543 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,546 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,550 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,554 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,559 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,562 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,567 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,570 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:28,576 creating build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,578 copying colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,581 copying colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,585 copying colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,588 copying colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,593 copying colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,597 copying colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,601 copying colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,605 copying colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,608 copying colossalai/auto_parallel/meta_profiler/meta_registry/where.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,611 copying colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,615 copying colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:28,620 creating build/lib/colossalai/cli/benchmark 2023-09-06T15:57:28,622 copying colossalai/cli/benchmark/models.py -> build/lib/colossalai/cli/benchmark 2023-09-06T15:57:28,625 copying colossalai/cli/benchmark/__init__.py -> build/lib/colossalai/cli/benchmark 2023-09-06T15:57:28,629 copying colossalai/cli/benchmark/benchmark.py -> build/lib/colossalai/cli/benchmark 2023-09-06T15:57:28,632 copying colossalai/cli/benchmark/utils.py -> build/lib/colossalai/cli/benchmark 2023-09-06T15:57:28,638 creating build/lib/colossalai/cli/check 2023-09-06T15:57:28,640 copying colossalai/cli/check/check_installation.py -> build/lib/colossalai/cli/check 2023-09-06T15:57:28,644 copying colossalai/cli/check/__init__.py -> build/lib/colossalai/cli/check 2023-09-06T15:57:28,649 creating build/lib/colossalai/cli/launcher 2023-09-06T15:57:28,650 copying colossalai/cli/launcher/hostinfo.py -> build/lib/colossalai/cli/launcher 2023-09-06T15:57:28,654 copying colossalai/cli/launcher/__init__.py -> build/lib/colossalai/cli/launcher 2023-09-06T15:57:28,657 copying colossalai/cli/launcher/run.py -> build/lib/colossalai/cli/launcher 2023-09-06T15:57:28,661 copying colossalai/cli/launcher/multinode_runner.py -> build/lib/colossalai/cli/launcher 2023-09-06T15:57:28,668 creating build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,669 copying colossalai/fx/tracer/registry.py -> build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,673 copying colossalai/fx/tracer/experimental.py -> build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,678 copying colossalai/fx/tracer/__init__.py -> build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,681 copying colossalai/fx/tracer/_symbolic_trace.py -> build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,684 copying colossalai/fx/tracer/_meta_trace.py -> build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,688 copying colossalai/fx/tracer/_tracer_utils.py -> build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,691 copying colossalai/fx/tracer/tracer.py -> build/lib/colossalai/fx/tracer 2023-09-06T15:57:28,698 creating build/lib/colossalai/fx/codegen 2023-09-06T15:57:28,699 copying colossalai/fx/codegen/__init__.py -> build/lib/colossalai/fx/codegen 2023-09-06T15:57:28,703 copying colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/lib/colossalai/fx/codegen 2023-09-06T15:57:28,711 creating build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,713 copying colossalai/fx/profiler/opcount.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,718 copying colossalai/fx/profiler/profiler.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,722 copying colossalai/fx/profiler/dataflow.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,726 copying colossalai/fx/profiler/memory_utils.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,730 copying colossalai/fx/profiler/tensor.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,733 copying colossalai/fx/profiler/__init__.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,737 copying colossalai/fx/profiler/shard_utils.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,741 copying colossalai/fx/profiler/constants.py -> build/lib/colossalai/fx/profiler 2023-09-06T15:57:28,746 creating build/lib/colossalai/fx/passes 2023-09-06T15:57:28,747 copying colossalai/fx/passes/shard_1d_pass.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,752 copying colossalai/fx/passes/adding_split_node_pass.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,756 copying colossalai/fx/passes/meta_info_prop.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,761 copying colossalai/fx/passes/passes_for_gpt2_test.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,766 copying colossalai/fx/passes/__init__.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,769 copying colossalai/fx/passes/concrete_info_prop.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,773 copying colossalai/fx/passes/split_module.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,777 copying colossalai/fx/passes/utils.py -> build/lib/colossalai/fx/passes 2023-09-06T15:57:28,782 creating build/lib/colossalai/fx/tracer/bias_addition_patch 2023-09-06T15:57:28,784 copying colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch 2023-09-06T15:57:28,789 creating build/lib/colossalai/fx/tracer/meta_patch 2023-09-06T15:57:28,790 copying colossalai/fx/tracer/meta_patch/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch 2023-09-06T15:57:28,796 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:28,797 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:28,801 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:28,805 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:28,808 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:28,812 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:28,817 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:28,819 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:28,823 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:28,826 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:28,830 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:28,835 creating build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,837 copying colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,840 copying colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,844 copying colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,847 copying colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,851 copying colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,855 copying colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,858 copying colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,862 copying colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:28,868 creating build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,869 copying colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,873 copying colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,877 copying colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,880 copying colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,884 copying colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,888 copying colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,891 copying colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,894 copying colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:28,900 creating build/lib/colossalai/fx/profiler/experimental 2023-09-06T15:57:28,902 copying colossalai/fx/profiler/experimental/profiler.py -> build/lib/colossalai/fx/profiler/experimental 2023-09-06T15:57:28,906 copying colossalai/fx/profiler/experimental/registry.py -> build/lib/colossalai/fx/profiler/experimental 2023-09-06T15:57:28,910 copying colossalai/fx/profiler/experimental/__init__.py -> build/lib/colossalai/fx/profiler/experimental 2023-09-06T15:57:28,913 copying colossalai/fx/profiler/experimental/shard_utils.py -> build/lib/colossalai/fx/profiler/experimental 2023-09-06T15:57:28,916 copying colossalai/fx/profiler/experimental/constants.py -> build/lib/colossalai/fx/profiler/experimental 2023-09-06T15:57:28,922 creating build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,924 copying colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,927 copying colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,931 copying colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,934 copying colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,938 copying colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,941 copying colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,944 copying colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,949 copying colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,952 copying colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,956 copying colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,959 copying colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:28,964 creating build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,966 copying colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,970 copying colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,973 copying colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,977 copying colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,980 copying colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,983 copying colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,987 copying colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,990 copying colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,994 copying colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:28,999 creating build/lib/colossalai/pipeline/middleware 2023-09-06T15:57:29,000 copying colossalai/pipeline/middleware/__init__.py -> build/lib/colossalai/pipeline/middleware 2023-09-06T15:57:29,004 copying colossalai/pipeline/middleware/topo.py -> build/lib/colossalai/pipeline/middleware 2023-09-06T15:57:29,010 creating build/lib/colossalai/pipeline/schedule 2023-09-06T15:57:29,011 copying colossalai/pipeline/schedule/__init__.py -> build/lib/colossalai/pipeline/schedule 2023-09-06T15:57:29,015 copying colossalai/pipeline/schedule/one_f_one_b.py -> build/lib/colossalai/pipeline/schedule 2023-09-06T15:57:29,020 copying colossalai/pipeline/schedule/base.py -> build/lib/colossalai/pipeline/schedule 2023-09-06T15:57:29,023 copying colossalai/pipeline/schedule/interleaved_pp.py -> build/lib/colossalai/pipeline/schedule 2023-09-06T15:57:29,028 copying colossalai/pipeline/schedule/_utils.py -> build/lib/colossalai/pipeline/schedule 2023-09-06T15:57:29,034 creating build/lib/colossalai/pipeline/rpc 2023-09-06T15:57:29,036 copying colossalai/pipeline/rpc/_pipeline_schedule.py -> build/lib/colossalai/pipeline/rpc 2023-09-06T15:57:29,041 copying colossalai/pipeline/rpc/__init__.py -> build/lib/colossalai/pipeline/rpc 2023-09-06T15:57:29,044 copying colossalai/pipeline/rpc/_pipeline_base.py -> build/lib/colossalai/pipeline/rpc 2023-09-06T15:57:29,052 copying colossalai/pipeline/rpc/utils.py -> build/lib/colossalai/pipeline/rpc 2023-09-06T15:57:29,057 creating build/lib/colossalai/pipeline/middleware/adaptor 2023-09-06T15:57:29,059 copying colossalai/pipeline/middleware/adaptor/__init__.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-09-06T15:57:29,063 copying colossalai/pipeline/middleware/adaptor/fx.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-09-06T15:57:29,070 creating build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,072 copying colossalai/tensor/d_tensor/sharding_spec.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,077 copying colossalai/tensor/d_tensor/layout.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,080 copying colossalai/tensor/d_tensor/__init__.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,084 copying colossalai/tensor/d_tensor/layout_converter.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,090 copying colossalai/tensor/d_tensor/api.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,095 copying colossalai/tensor/d_tensor/misc.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,099 copying colossalai/tensor/d_tensor/utils.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,102 copying colossalai/tensor/d_tensor/comm_spec.py -> build/lib/colossalai/tensor/d_tensor 2023-09-06T15:57:29,110 creating build/lib/colossalai/_analyzer/fx 2023-09-06T15:57:29,111 copying colossalai/_analyzer/fx/__init__.py -> build/lib/colossalai/_analyzer/fx 2023-09-06T15:57:29,115 copying colossalai/_analyzer/fx/node_util.py -> build/lib/colossalai/_analyzer/fx 2023-09-06T15:57:29,119 copying colossalai/_analyzer/fx/codegen.py -> build/lib/colossalai/_analyzer/fx 2023-09-06T15:57:29,124 copying colossalai/_analyzer/fx/symbolic_profile.py -> build/lib/colossalai/_analyzer/fx 2023-09-06T15:57:29,128 copying colossalai/_analyzer/fx/graph_module.py -> build/lib/colossalai/_analyzer/fx 2023-09-06T15:57:29,133 creating build/lib/colossalai/_analyzer/_subclasses 2023-09-06T15:57:29,135 copying colossalai/_analyzer/_subclasses/_meta_registration.py -> build/lib/colossalai/_analyzer/_subclasses 2023-09-06T15:57:29,141 copying colossalai/_analyzer/_subclasses/__init__.py -> build/lib/colossalai/_analyzer/_subclasses 2023-09-06T15:57:29,144 copying colossalai/_analyzer/_subclasses/flop_tensor.py -> build/lib/colossalai/_analyzer/_subclasses 2023-09-06T15:57:29,149 copying colossalai/_analyzer/_subclasses/meta_tensor.py -> build/lib/colossalai/_analyzer/_subclasses 2023-09-06T15:57:29,153 copying colossalai/_analyzer/_subclasses/_monkey_patch.py -> build/lib/colossalai/_analyzer/_subclasses 2023-09-06T15:57:29,159 creating build/lib/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:29,160 copying colossalai/_analyzer/fx/tracer/symbolic_trace.py -> build/lib/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:29,165 copying colossalai/_analyzer/fx/tracer/__init__.py -> build/lib/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:29,168 copying colossalai/_analyzer/fx/tracer/bias_addition.py -> build/lib/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:29,172 copying colossalai/_analyzer/fx/tracer/proxy.py -> build/lib/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:29,176 copying colossalai/_analyzer/fx/tracer/custom_leaf_module.py -> build/lib/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:29,179 copying colossalai/_analyzer/fx/tracer/tracer.py -> build/lib/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:29,185 creating build/lib/colossalai/_analyzer/fx/passes 2023-09-06T15:57:29,187 copying colossalai/_analyzer/fx/passes/__init__.py -> build/lib/colossalai/_analyzer/fx/passes 2023-09-06T15:57:29,191 copying colossalai/_analyzer/fx/passes/shape_prop.py -> build/lib/colossalai/_analyzer/fx/passes 2023-09-06T15:57:29,196 copying colossalai/_analyzer/fx/passes/graph_profile.py -> build/lib/colossalai/_analyzer/fx/passes 2023-09-06T15:57:29,203 creating build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,204 copying colossalai/booster/plugin/gemini_plugin.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,209 copying colossalai/booster/plugin/low_level_zero_plugin.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,214 copying colossalai/booster/plugin/__init__.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,218 copying colossalai/booster/plugin/dp_plugin_base.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,222 copying colossalai/booster/plugin/torch_fsdp_plugin.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,226 copying colossalai/booster/plugin/pp_plugin_base.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,230 copying colossalai/booster/plugin/torch_ddp_plugin.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,234 copying colossalai/booster/plugin/plugin_base.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,238 copying colossalai/booster/plugin/hybrid_parallel_plugin.py -> build/lib/colossalai/booster/plugin 2023-09-06T15:57:29,245 creating build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,246 copying colossalai/booster/mixed_precision/fp8.py -> build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,250 copying colossalai/booster/mixed_precision/__init__.py -> build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,253 copying colossalai/booster/mixed_precision/bf16.py -> build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,257 copying colossalai/booster/mixed_precision/fp16_naive.py -> build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,260 copying colossalai/booster/mixed_precision/fp16_torch.py -> build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,264 copying colossalai/booster/mixed_precision/mixed_precision_base.py -> build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,268 copying colossalai/booster/mixed_precision/fp16_apex.py -> build/lib/colossalai/booster/mixed_precision 2023-09-06T15:57:29,273 creating build/lib/colossalai/legacy/builder 2023-09-06T15:57:29,274 copying colossalai/legacy/builder/__init__.py -> build/lib/colossalai/legacy/builder 2023-09-06T15:57:29,279 copying colossalai/legacy/builder/builder.py -> build/lib/colossalai/legacy/builder 2023-09-06T15:57:29,284 creating build/lib/colossalai/legacy/registry 2023-09-06T15:57:29,286 copying colossalai/legacy/registry/registry.py -> build/lib/colossalai/legacy/registry 2023-09-06T15:57:29,290 copying colossalai/legacy/registry/__init__.py -> build/lib/colossalai/legacy/registry 2023-09-06T15:57:29,294 creating build/lib/colossalai/legacy/trainer 2023-09-06T15:57:29,297 copying colossalai/legacy/trainer/_trainer.py -> build/lib/colossalai/legacy/trainer 2023-09-06T15:57:29,302 copying colossalai/legacy/trainer/__init__.py -> build/lib/colossalai/legacy/trainer 2023-09-06T15:57:29,307 creating build/lib/colossalai/legacy/engine 2023-09-06T15:57:29,308 copying colossalai/legacy/engine/__init__.py -> build/lib/colossalai/legacy/engine 2023-09-06T15:57:29,312 copying colossalai/legacy/engine/_base_engine.py -> build/lib/colossalai/legacy/engine 2023-09-06T15:57:29,318 creating build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,320 copying colossalai/legacy/trainer/hooks/_base_hook.py -> build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,325 copying colossalai/legacy/trainer/hooks/_log_hook.py -> build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,329 copying colossalai/legacy/trainer/hooks/__init__.py -> build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,333 copying colossalai/legacy/trainer/hooks/_metric_hook.py -> build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,338 copying colossalai/legacy/trainer/hooks/_commons_.py -> build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,341 copying colossalai/legacy/trainer/hooks/_lr_scheduler_hook.py -> build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,345 copying colossalai/legacy/trainer/hooks/_checkpoint_hook.py -> build/lib/colossalai/legacy/trainer/hooks 2023-09-06T15:57:29,351 creating build/lib/colossalai/legacy/engine/schedule 2023-09-06T15:57:29,353 copying colossalai/legacy/engine/schedule/_base_schedule.py -> build/lib/colossalai/legacy/engine/schedule 2023-09-06T15:57:29,358 copying colossalai/legacy/engine/schedule/_pipeline_schedule.py -> build/lib/colossalai/legacy/engine/schedule 2023-09-06T15:57:29,364 copying colossalai/legacy/engine/schedule/__init__.py -> build/lib/colossalai/legacy/engine/schedule 2023-09-06T15:57:29,368 copying colossalai/legacy/engine/schedule/_pipeline_schedule_v2.py -> build/lib/colossalai/legacy/engine/schedule 2023-09-06T15:57:29,372 copying colossalai/legacy/engine/schedule/_non_pipeline_schedule.py -> build/lib/colossalai/legacy/engine/schedule 2023-09-06T15:57:29,377 creating build/lib/colossalai/legacy/engine/gradient_accumulation 2023-09-06T15:57:29,379 copying colossalai/legacy/engine/gradient_accumulation/_gradient_accumulation.py -> build/lib/colossalai/legacy/engine/gradient_accumulation 2023-09-06T15:57:29,384 copying colossalai/legacy/engine/gradient_accumulation/__init__.py -> build/lib/colossalai/legacy/engine/gradient_accumulation 2023-09-06T15:57:29,390 creating build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,392 copying colossalai/legacy/engine/gradient_handler/_moe_gradient_handler.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,396 copying colossalai/legacy/engine/gradient_handler/_base_gradient_handler.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,399 copying colossalai/legacy/engine/gradient_handler/__init__.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,403 copying colossalai/legacy/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,407 copying colossalai/legacy/engine/gradient_handler/_zero_gradient_handler.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,410 copying colossalai/legacy/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,414 copying colossalai/legacy/engine/gradient_handler/utils.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,418 copying colossalai/legacy/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/lib/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:29,423 creating build/lib/colossalai/zero/low_level 2023-09-06T15:57:29,425 copying colossalai/zero/low_level/low_level_optim.py -> build/lib/colossalai/zero/low_level 2023-09-06T15:57:29,431 copying colossalai/zero/low_level/__init__.py -> build/lib/colossalai/zero/low_level 2023-09-06T15:57:29,435 copying colossalai/zero/low_level/_utils.py -> build/lib/colossalai/zero/low_level 2023-09-06T15:57:29,440 creating build/lib/colossalai/zero/legacy 2023-09-06T15:57:29,442 copying colossalai/zero/legacy/__init__.py -> build/lib/colossalai/zero/legacy 2023-09-06T15:57:29,448 creating build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,450 copying colossalai/zero/gemini/gemini_optimizer.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,456 copying colossalai/zero/gemini/placement_policy.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,460 copying colossalai/zero/gemini/__init__.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,464 copying colossalai/zero/gemini/colo_init_context.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,468 copying colossalai/zero/gemini/gemini_hook.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,472 copying colossalai/zero/gemini/gemini_mgr.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,476 copying colossalai/zero/gemini/utils.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,480 copying colossalai/zero/gemini/gemini_ddp.py -> build/lib/colossalai/zero/gemini 2023-09-06T15:57:29,489 creating build/lib/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:29,491 copying colossalai/zero/low_level/bookkeeping/bucket_store.py -> build/lib/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:29,495 copying colossalai/zero/low_level/bookkeeping/__init__.py -> build/lib/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:29,499 copying colossalai/zero/low_level/bookkeeping/parameter_store.py -> build/lib/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:29,502 copying colossalai/zero/low_level/bookkeeping/tensor_bucket.py -> build/lib/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:29,506 copying colossalai/zero/low_level/bookkeeping/gradient_store.py -> build/lib/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:29,510 copying colossalai/zero/low_level/bookkeeping/base_store.py -> build/lib/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:29,515 creating build/lib/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:29,517 copying colossalai/zero/legacy/sharded_model/__init__.py -> build/lib/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:29,520 copying colossalai/zero/legacy/sharded_model/utils.py -> build/lib/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:29,524 copying colossalai/zero/legacy/sharded_model/reduce_scatter.py -> build/lib/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:29,529 copying colossalai/zero/legacy/sharded_model/sharded_model_v2.py -> build/lib/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:29,534 copying colossalai/zero/legacy/sharded_model/_utils.py -> build/lib/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:29,538 copying colossalai/zero/legacy/sharded_model/zero_hook.py -> build/lib/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:29,543 creating build/lib/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:29,545 copying colossalai/zero/legacy/shard_utils/base_shard_strategy.py -> build/lib/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:29,549 copying colossalai/zero/legacy/shard_utils/__init__.py -> build/lib/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:29,552 copying colossalai/zero/legacy/shard_utils/bucket_tensor_shard_strategy.py -> build/lib/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:29,556 copying colossalai/zero/legacy/shard_utils/commons.py -> build/lib/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:29,560 copying colossalai/zero/legacy/shard_utils/tensor_shard_strategy.py -> build/lib/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:29,565 creating build/lib/colossalai/zero/legacy/init_ctx 2023-09-06T15:57:29,566 copying colossalai/zero/legacy/init_ctx/init_context.py -> build/lib/colossalai/zero/legacy/init_ctx 2023-09-06T15:57:29,571 copying colossalai/zero/legacy/init_ctx/__init__.py -> build/lib/colossalai/zero/legacy/init_ctx 2023-09-06T15:57:29,576 creating build/lib/colossalai/zero/legacy/gemini 2023-09-06T15:57:29,578 copying colossalai/zero/legacy/gemini/tensor_utils.py -> build/lib/colossalai/zero/legacy/gemini 2023-09-06T15:57:29,582 copying colossalai/zero/legacy/gemini/gemini_context.py -> build/lib/colossalai/zero/legacy/gemini 2023-09-06T15:57:29,586 copying colossalai/zero/legacy/gemini/stateful_tensor_mgr.py -> build/lib/colossalai/zero/legacy/gemini 2023-09-06T15:57:29,590 copying colossalai/zero/legacy/gemini/__init__.py -> build/lib/colossalai/zero/legacy/gemini 2023-09-06T15:57:29,594 copying colossalai/zero/legacy/gemini/stateful_tensor.py -> build/lib/colossalai/zero/legacy/gemini 2023-09-06T15:57:29,598 copying colossalai/zero/legacy/gemini/tensor_placement_policy.py -> build/lib/colossalai/zero/legacy/gemini 2023-09-06T15:57:29,604 creating build/lib/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:29,606 copying colossalai/zero/legacy/sharded_param/sharded_tensor.py -> build/lib/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:29,610 copying colossalai/zero/legacy/sharded_param/sharded_param.py -> build/lib/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:29,613 copying colossalai/zero/legacy/sharded_param/__init__.py -> build/lib/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:29,618 creating build/lib/colossalai/zero/legacy/sharded_optim 2023-09-06T15:57:29,620 copying colossalai/zero/legacy/sharded_optim/__init__.py -> build/lib/colossalai/zero/legacy/sharded_optim 2023-09-06T15:57:29,624 copying colossalai/zero/legacy/sharded_optim/sharded_optim_v2.py -> build/lib/colossalai/zero/legacy/sharded_optim 2023-09-06T15:57:29,630 creating build/lib/colossalai/zero/legacy/gemini/paramhooks 2023-09-06T15:57:29,632 copying colossalai/zero/legacy/gemini/paramhooks/_param_hookmgr.py -> build/lib/colossalai/zero/legacy/gemini/paramhooks 2023-09-06T15:57:29,636 copying colossalai/zero/legacy/gemini/paramhooks/__init__.py -> build/lib/colossalai/zero/legacy/gemini/paramhooks 2023-09-06T15:57:29,641 creating build/lib/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:29,642 copying colossalai/zero/legacy/gemini/ophooks/__init__.py -> build/lib/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:29,647 copying colossalai/zero/legacy/gemini/ophooks/runtime_mem_tracer_hook.py -> build/lib/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:29,651 copying colossalai/zero/legacy/gemini/ophooks/_shard_grad_ophook.py -> build/lib/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:29,655 copying colossalai/zero/legacy/gemini/ophooks/utils.py -> build/lib/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:29,659 copying colossalai/zero/legacy/gemini/ophooks/_shard_param_ophook.py -> build/lib/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:29,665 creating build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,667 copying colossalai/zero/gemini/memory_tracer/memory_monitor.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,671 copying colossalai/zero/gemini/memory_tracer/param_runtime_order.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,674 copying colossalai/zero/gemini/memory_tracer/__init__.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,678 copying colossalai/zero/gemini/memory_tracer/runtime_mem_tracer.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,682 copying colossalai/zero/gemini/memory_tracer/memstats_collector.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,686 copying colossalai/zero/gemini/memory_tracer/static_memstats_collector.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,690 copying colossalai/zero/gemini/memory_tracer/memory_stats.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,695 copying colossalai/zero/gemini/memory_tracer/utils.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,698 copying colossalai/zero/gemini/memory_tracer/chunk_memstats_collector.py -> build/lib/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:29,703 creating build/lib/colossalai/zero/gemini/chunk 2023-09-06T15:57:29,705 copying colossalai/zero/gemini/chunk/__init__.py -> build/lib/colossalai/zero/gemini/chunk 2023-09-06T15:57:29,709 copying colossalai/zero/gemini/chunk/chunk.py -> build/lib/colossalai/zero/gemini/chunk 2023-09-06T15:57:29,715 copying colossalai/zero/gemini/chunk/manager.py -> build/lib/colossalai/zero/gemini/chunk 2023-09-06T15:57:29,719 copying colossalai/zero/gemini/chunk/search_utils.py -> build/lib/colossalai/zero/gemini/chunk 2023-09-06T15:57:29,723 copying colossalai/zero/gemini/chunk/utils.py -> build/lib/colossalai/zero/gemini/chunk 2023-09-06T15:57:29,729 creating build/lib/colossalai/nn/loss 2023-09-06T15:57:29,730 copying colossalai/nn/loss/loss_moe.py -> build/lib/colossalai/nn/loss 2023-09-06T15:57:29,734 copying colossalai/nn/loss/__init__.py -> build/lib/colossalai/nn/loss 2023-09-06T15:57:29,738 copying colossalai/nn/loss/loss_2d.py -> build/lib/colossalai/nn/loss 2023-09-06T15:57:29,742 copying colossalai/nn/loss/loss_1d.py -> build/lib/colossalai/nn/loss 2023-09-06T15:57:29,747 copying colossalai/nn/loss/loss_2p5d.py -> build/lib/colossalai/nn/loss 2023-09-06T15:57:29,751 copying colossalai/nn/loss/loss_3d.py -> build/lib/colossalai/nn/loss 2023-09-06T15:57:29,757 creating build/lib/colossalai/nn/layer 2023-09-06T15:57:29,758 copying colossalai/nn/layer/__init__.py -> build/lib/colossalai/nn/layer 2023-09-06T15:57:29,762 copying colossalai/nn/layer/base_layer.py -> build/lib/colossalai/nn/layer 2023-09-06T15:57:29,767 creating build/lib/colossalai/nn/metric 2023-09-06T15:57:29,769 copying colossalai/nn/metric/__init__.py -> build/lib/colossalai/nn/metric 2023-09-06T15:57:29,773 copying colossalai/nn/metric/accuracy_2p5d.py -> build/lib/colossalai/nn/metric 2023-09-06T15:57:29,777 copying colossalai/nn/metric/accuracy_3d.py -> build/lib/colossalai/nn/metric 2023-09-06T15:57:29,780 copying colossalai/nn/metric/accuracy_2d.py -> build/lib/colossalai/nn/metric 2023-09-06T15:57:29,784 copying colossalai/nn/metric/_utils.py -> build/lib/colossalai/nn/metric 2023-09-06T15:57:29,790 creating build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,792 copying colossalai/nn/lr_scheduler/torch.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,796 copying colossalai/nn/lr_scheduler/__init__.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,799 copying colossalai/nn/lr_scheduler/linear.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,803 copying colossalai/nn/lr_scheduler/onecycle.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,807 copying colossalai/nn/lr_scheduler/cosine.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,811 copying colossalai/nn/lr_scheduler/delayed.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,816 copying colossalai/nn/lr_scheduler/poly.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,820 copying colossalai/nn/lr_scheduler/multistep.py -> build/lib/colossalai/nn/lr_scheduler 2023-09-06T15:57:29,826 creating build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,827 copying colossalai/nn/_ops/__init__.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,832 copying colossalai/nn/_ops/batch_norm.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,835 copying colossalai/nn/_ops/addmm.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,840 copying colossalai/nn/_ops/linear.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,844 copying colossalai/nn/_ops/layernorm.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,848 copying colossalai/nn/_ops/loss.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,851 copying colossalai/nn/_ops/embedding_bag.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,856 copying colossalai/nn/_ops/element_wise.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,860 copying colossalai/nn/_ops/embedding.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,865 copying colossalai/nn/_ops/view.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,868 copying colossalai/nn/_ops/_utils.py -> build/lib/colossalai/nn/_ops 2023-09-06T15:57:29,874 creating build/lib/colossalai/nn/parallel 2023-09-06T15:57:29,876 copying colossalai/nn/parallel/data_parallel.py -> build/lib/colossalai/nn/parallel 2023-09-06T15:57:29,881 copying colossalai/nn/parallel/__init__.py -> build/lib/colossalai/nn/parallel 2023-09-06T15:57:29,884 copying colossalai/nn/parallel/reducer.py -> build/lib/colossalai/nn/parallel 2023-09-06T15:57:29,890 creating build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,892 copying colossalai/nn/optimizer/fused_lamb.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,897 copying colossalai/nn/optimizer/lamb.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,901 copying colossalai/nn/optimizer/cpu_adam.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,906 copying colossalai/nn/optimizer/__init__.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,909 copying colossalai/nn/optimizer/fused_adam.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,913 copying colossalai/nn/optimizer/hybrid_adam.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,918 copying colossalai/nn/optimizer/fused_sgd.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,922 copying colossalai/nn/optimizer/lars.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,926 copying colossalai/nn/optimizer/colossalai_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,930 copying colossalai/nn/optimizer/nvme_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-09-06T15:57:29,936 creating build/lib/colossalai/nn/layer/vanilla 2023-09-06T15:57:29,937 copying colossalai/nn/layer/vanilla/__init__.py -> build/lib/colossalai/nn/layer/vanilla 2023-09-06T15:57:29,941 copying colossalai/nn/layer/vanilla/layers.py -> build/lib/colossalai/nn/layer/vanilla 2023-09-06T15:57:29,948 creating build/lib/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:29,949 copying colossalai/nn/layer/parallel_sequence/__init__.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:29,953 copying colossalai/nn/layer/parallel_sequence/_operation.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:29,958 copying colossalai/nn/layer/parallel_sequence/layers.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:29,962 copying colossalai/nn/layer/parallel_sequence/_utils.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:29,968 creating build/lib/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:29,970 copying colossalai/nn/layer/colossalai_layer/dropout.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:29,973 copying colossalai/nn/layer/colossalai_layer/__init__.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:29,977 copying colossalai/nn/layer/colossalai_layer/linear.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:29,981 copying colossalai/nn/layer/colossalai_layer/normalization.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:29,985 copying colossalai/nn/layer/colossalai_layer/embedding.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:29,989 copying colossalai/nn/layer/colossalai_layer/_utils.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:29,994 creating build/lib/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:29,997 copying colossalai/nn/layer/parallel_2p5d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:30,001 copying colossalai/nn/layer/parallel_2p5d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:30,007 copying colossalai/nn/layer/parallel_2p5d/layers.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:30,014 copying colossalai/nn/layer/parallel_2p5d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:30,019 creating build/lib/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:30,021 copying colossalai/nn/layer/parallel_3d/__init__.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:30,025 copying colossalai/nn/layer/parallel_3d/_operation.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:30,030 copying colossalai/nn/layer/parallel_3d/layers.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:30,036 copying colossalai/nn/layer/parallel_3d/_utils.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:30,042 creating build/lib/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:30,043 copying colossalai/nn/layer/parallel_2d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:30,047 copying colossalai/nn/layer/parallel_2d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:30,054 copying colossalai/nn/layer/parallel_2d/layers.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:30,061 copying colossalai/nn/layer/parallel_2d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:30,067 creating build/lib/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:30,069 copying colossalai/nn/layer/parallel_1d/__init__.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:30,073 copying colossalai/nn/layer/parallel_1d/_operation.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:30,077 copying colossalai/nn/layer/parallel_1d/layers.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:30,084 copying colossalai/nn/layer/parallel_1d/_utils.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:30,090 creating build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,092 copying colossalai/nn/layer/moe/experts.py -> build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,098 copying colossalai/nn/layer/moe/checkpoint.py -> build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,101 copying colossalai/nn/layer/moe/__init__.py -> build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,104 copying colossalai/nn/layer/moe/_operation.py -> build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,109 copying colossalai/nn/layer/moe/layers.py -> build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,114 copying colossalai/nn/layer/moe/utils.py -> build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,118 copying colossalai/nn/layer/moe/routers.py -> build/lib/colossalai/nn/layer/moe 2023-09-06T15:57:30,123 creating build/lib/colossalai/nn/layer/wrapper 2023-09-06T15:57:30,125 copying colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/lib/colossalai/nn/layer/wrapper 2023-09-06T15:57:30,129 copying colossalai/nn/layer/wrapper/__init__.py -> build/lib/colossalai/nn/layer/wrapper 2023-09-06T15:57:30,134 creating build/lib/colossalai/nn/layer/utils 2023-09-06T15:57:30,136 copying colossalai/nn/layer/utils/__init__.py -> build/lib/colossalai/nn/layer/utils 2023-09-06T15:57:30,140 copying colossalai/nn/layer/utils/common.py -> build/lib/colossalai/nn/layer/utils 2023-09-06T15:57:30,145 creating build/lib/colossalai/nn/parallel/layers 2023-09-06T15:57:30,147 copying colossalai/nn/parallel/layers/colo_module.py -> build/lib/colossalai/nn/parallel/layers 2023-09-06T15:57:30,151 copying colossalai/nn/parallel/layers/module_utils.py -> build/lib/colossalai/nn/parallel/layers 2023-09-06T15:57:30,155 copying colossalai/nn/parallel/layers/__init__.py -> build/lib/colossalai/nn/parallel/layers 2023-09-06T15:57:30,158 copying colossalai/nn/parallel/layers/linear.py -> build/lib/colossalai/nn/parallel/layers 2023-09-06T15:57:30,162 copying colossalai/nn/parallel/layers/embedding.py -> build/lib/colossalai/nn/parallel/layers 2023-09-06T15:57:30,168 creating build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,170 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,175 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,180 copying colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,183 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,188 copying colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,193 copying colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,197 copying colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,202 copying colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,206 copying colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:30,214 creating build/lib/colossalai/shardformer/shard 2023-09-06T15:57:30,216 copying colossalai/shardformer/shard/shard_config.py -> build/lib/colossalai/shardformer/shard 2023-09-06T15:57:30,221 copying colossalai/shardformer/shard/__init__.py -> build/lib/colossalai/shardformer/shard 2023-09-06T15:57:30,225 copying colossalai/shardformer/shard/shardformer.py -> build/lib/colossalai/shardformer/shard 2023-09-06T15:57:30,229 copying colossalai/shardformer/shard/utils.py -> build/lib/colossalai/shardformer/shard 2023-09-06T15:57:30,233 copying colossalai/shardformer/shard/sharder.py -> build/lib/colossalai/shardformer/shard 2023-09-06T15:57:30,239 creating build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,241 copying colossalai/shardformer/layer/dropout.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,245 copying colossalai/shardformer/layer/__init__.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,249 copying colossalai/shardformer/layer/_operation.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,254 copying colossalai/shardformer/layer/linear.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,259 copying colossalai/shardformer/layer/loss.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,263 copying colossalai/shardformer/layer/normalization.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,267 copying colossalai/shardformer/layer/parallel_module.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,272 copying colossalai/shardformer/layer/embedding.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,276 copying colossalai/shardformer/layer/utils.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,281 copying colossalai/shardformer/layer/qkv_fused_linear.py -> build/lib/colossalai/shardformer/layer 2023-09-06T15:57:30,289 creating build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,291 copying colossalai/shardformer/policies/llama.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,296 copying colossalai/shardformer/policies/vit.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,301 copying colossalai/shardformer/policies/t5.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,307 copying colossalai/shardformer/policies/base_policy.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,312 copying colossalai/shardformer/policies/__init__.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,315 copying colossalai/shardformer/policies/sam.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,320 copying colossalai/shardformer/policies/gpt2.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,325 copying colossalai/shardformer/policies/opt.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,330 copying colossalai/shardformer/policies/bert.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,336 copying colossalai/shardformer/policies/chatglm2.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,342 copying colossalai/shardformer/policies/whisper.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,349 copying colossalai/shardformer/policies/auto_policy.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,355 copying colossalai/shardformer/policies/blip2.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,360 copying colossalai/shardformer/policies/bloom.py -> build/lib/colossalai/shardformer/policies 2023-09-06T15:57:30,369 creating build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,371 copying colossalai/shardformer/modeling/llama.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,377 copying colossalai/shardformer/modeling/vit.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,383 copying colossalai/shardformer/modeling/t5.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,391 copying colossalai/shardformer/modeling/__init__.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,395 copying colossalai/shardformer/modeling/sam.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,401 copying colossalai/shardformer/modeling/gpt2.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,408 copying colossalai/shardformer/modeling/jit.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,412 copying colossalai/shardformer/modeling/opt.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,418 copying colossalai/shardformer/modeling/bert.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,427 copying colossalai/shardformer/modeling/chatglm2.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,432 copying colossalai/shardformer/modeling/whisper.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,438 copying colossalai/shardformer/modeling/blip2.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,442 copying colossalai/shardformer/modeling/bloom.py -> build/lib/colossalai/shardformer/modeling 2023-09-06T15:57:30,451 creating build/lib/colossalai/utils/data_sampler 2023-09-06T15:57:30,454 copying colossalai/utils/data_sampler/data_parallel_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-09-06T15:57:30,459 copying colossalai/utils/data_sampler/__init__.py -> build/lib/colossalai/utils/data_sampler 2023-09-06T15:57:30,463 copying colossalai/utils/data_sampler/base_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-09-06T15:57:30,468 creating build/lib/colossalai/utils/tensor_detector 2023-09-06T15:57:30,471 copying colossalai/utils/tensor_detector/tensor_detector.py -> build/lib/colossalai/utils/tensor_detector 2023-09-06T15:57:30,476 copying colossalai/utils/tensor_detector/__init__.py -> build/lib/colossalai/utils/tensor_detector 2023-09-06T15:57:30,482 creating build/lib/colossalai/utils/model 2023-09-06T15:57:30,484 copying colossalai/utils/model/__init__.py -> build/lib/colossalai/utils/model 2023-09-06T15:57:30,488 copying colossalai/utils/model/utils.py -> build/lib/colossalai/utils/model 2023-09-06T15:57:30,494 creating build/lib/colossalai/utils/profiler 2023-09-06T15:57:30,497 copying colossalai/utils/profiler/profiler.py -> build/lib/colossalai/utils/profiler 2023-09-06T15:57:30,502 copying colossalai/utils/profiler/__init__.py -> build/lib/colossalai/utils/profiler 2023-09-06T15:57:30,506 copying colossalai/utils/profiler/extention.py -> build/lib/colossalai/utils/profiler 2023-09-06T15:57:30,511 copying colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/lib/colossalai/utils/profiler 2023-09-06T15:57:30,516 creating build/lib/colossalai/utils/multi_tensor_apply 2023-09-06T15:57:30,519 copying colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-09-06T15:57:30,524 copying colossalai/utils/multi_tensor_apply/__init__.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-09-06T15:57:30,529 creating build/lib/colossalai/utils/rank_recorder 2023-09-06T15:57:30,531 copying colossalai/utils/rank_recorder/__init__.py -> build/lib/colossalai/utils/rank_recorder 2023-09-06T15:57:30,536 copying colossalai/utils/rank_recorder/rank_recorder.py -> build/lib/colossalai/utils/rank_recorder 2023-09-06T15:57:30,543 creating build/lib/colossalai/utils/checkpoint 2023-09-06T15:57:30,545 copying colossalai/utils/checkpoint/__init__.py -> build/lib/colossalai/utils/checkpoint 2023-09-06T15:57:30,550 copying colossalai/utils/checkpoint/utils.py -> build/lib/colossalai/utils/checkpoint 2023-09-06T15:57:30,554 copying colossalai/utils/checkpoint/module_checkpoint.py -> build/lib/colossalai/utils/checkpoint 2023-09-06T15:57:30,560 creating build/lib/colossalai/utils/profiler/legacy 2023-09-06T15:57:30,562 copying colossalai/utils/profiler/legacy/prof_utils.py -> build/lib/colossalai/utils/profiler/legacy 2023-09-06T15:57:30,565 copying colossalai/utils/profiler/legacy/__init__.py -> build/lib/colossalai/utils/profiler/legacy 2023-09-06T15:57:30,572 copying colossalai/utils/profiler/legacy/comm_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-09-06T15:57:30,577 copying colossalai/utils/profiler/legacy/pcie_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-09-06T15:57:30,617 creating build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,619 copying colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,624 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,630 copying colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,635 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,639 copying colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,644 copying colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,649 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,655 copying colossalai/kernel/cuda_native/csrc/type_shim.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,660 copying colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,666 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,671 copying colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,675 copying colossalai/kernel/cuda_native/csrc/compat.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,680 copying colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,685 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,689 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,693 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,698 copying colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,703 copying colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,707 copying colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,713 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,719 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,723 copying colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,728 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:30,734 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels 2023-09-06T15:57:30,734 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,737 copying colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,741 copying colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,747 copying colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,751 copying colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,755 copying colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,759 copying colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,763 copying colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,767 copying colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,771 copying colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,775 copying colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,779 copying colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:30,783 copying colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,030 /usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2023-09-06T15:57:31,030 !! 2023-09-06T15:57:31,031 ******************************************************************************** 2023-09-06T15:57:31,031 Please avoid running ``setup.py`` directly. 2023-09-06T15:57:31,032 Instead, use pypa/build, pypa/installer or other 2023-09-06T15:57:31,032 standards-based tools. 2023-09-06T15:57:31,033 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2023-09-06T15:57:31,033 ******************************************************************************** 2023-09-06T15:57:31,034 !! 2023-09-06T15:57:31,034 self.initialize_options() 2023-09-06T15:57:31,087 installing to build/bdist.linux-armv7l/wheel 2023-09-06T15:57:31,087 running install 2023-09-06T15:57:31,146 running install_lib 2023-09-06T15:57:31,203 creating build/bdist.linux-armv7l 2023-09-06T15:57:31,204 creating build/bdist.linux-armv7l/wheel 2023-09-06T15:57:31,209 creating build/bdist.linux-armv7l/wheel/colossalai 2023-09-06T15:57:31,212 creating build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-09-06T15:57:31,215 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-09-06T15:57:31,217 copying build/lib/colossalai/kernel/cuda_native/layer_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-09-06T15:57:31,222 copying build/lib/colossalai/kernel/cuda_native/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-09-06T15:57:31,225 copying build/lib/colossalai/kernel/cuda_native/multihead_attention.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-09-06T15:57:31,231 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:31,233 copying build/lib/colossalai/kernel/cuda_native/mha/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:31,237 copying build/lib/colossalai/kernel/cuda_native/mha/mem_eff_attn.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:31,241 copying build/lib/colossalai/kernel/cuda_native/mha/mha.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:31,245 copying build/lib/colossalai/kernel/cuda_native/mha/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:31,250 copying build/lib/colossalai/kernel/cuda_native/mha/flash_attn_2.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/mha 2023-09-06T15:57:31,253 copying build/lib/colossalai/kernel/cuda_native/scaled_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-09-06T15:57:31,260 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,262 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,266 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,271 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,278 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels 2023-09-06T15:57:31,281 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,283 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,287 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,292 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,296 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,300 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,303 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,307 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,310 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,314 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,317 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,321 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,324 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-09-06T15:57:31,329 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,333 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,337 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,341 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,346 copying build/lib/colossalai/kernel/cuda_native/csrc/type_shim.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,351 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,355 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,359 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,362 copying build/lib/colossalai/kernel/cuda_native/csrc/compat.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,366 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,370 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,373 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,377 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,381 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,385 copying build/lib/colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,389 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,394 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,399 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,403 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,408 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-09-06T15:57:31,412 copying build/lib/colossalai/kernel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-09-06T15:57:31,417 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-09-06T15:57:31,418 copying build/lib/colossalai/kernel/jit/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-09-06T15:57:31,422 copying build/lib/colossalai/kernel/jit/bias_dropout_add.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-09-06T15:57:31,425 copying build/lib/colossalai/kernel/jit/option.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-09-06T15:57:31,429 copying build/lib/colossalai/kernel/jit/bias_gelu.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-09-06T15:57:31,434 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,435 copying build/lib/colossalai/kernel/op_builder/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,439 copying build/lib/colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,442 copying build/lib/colossalai/kernel/op_builder/fused_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,446 copying build/lib/colossalai/kernel/op_builder/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,449 copying build/lib/colossalai/kernel/op_builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,453 copying build/lib/colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,456 copying build/lib/colossalai/kernel/op_builder/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,460 copying build/lib/colossalai/kernel/op_builder/multi_head_attn.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,463 copying build/lib/colossalai/kernel/op_builder/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,467 copying build/lib/colossalai/kernel/op_builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-09-06T15:57:31,473 creating build/bdist.linux-armv7l/wheel/colossalai/context 2023-09-06T15:57:31,474 copying build/lib/colossalai/context/moe_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-09-06T15:57:31,479 copying build/lib/colossalai/context/parallel_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-09-06T15:57:31,484 copying build/lib/colossalai/context/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-09-06T15:57:31,488 creating build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-09-06T15:57:31,490 copying build/lib/colossalai/context/random/_helper.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-09-06T15:57:31,494 copying build/lib/colossalai/context/random/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-09-06T15:57:31,498 copying build/lib/colossalai/context/random/seed_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-09-06T15:57:31,501 copying build/lib/colossalai/context/parallel_mode.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-09-06T15:57:31,505 copying build/lib/colossalai/context/config.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-09-06T15:57:31,510 creating build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,512 copying build/lib/colossalai/context/process_group_initializer/initializer_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,517 copying build/lib/colossalai/context/process_group_initializer/initializer_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,520 copying build/lib/colossalai/context/process_group_initializer/initializer_data.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,523 copying build/lib/colossalai/context/process_group_initializer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,527 copying build/lib/colossalai/context/process_group_initializer/process_group_initializer.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,530 copying build/lib/colossalai/context/process_group_initializer/initializer_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,535 copying build/lib/colossalai/context/process_group_initializer/initializer_sequence.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,539 copying build/lib/colossalai/context/process_group_initializer/initializer_model.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,543 copying build/lib/colossalai/context/process_group_initializer/initializer_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,546 copying build/lib/colossalai/context/process_group_initializer/initializer_pipeline.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,550 copying build/lib/colossalai/context/process_group_initializer/initializer_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-09-06T15:57:31,555 copying build/lib/colossalai/context/singleton_meta.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-09-06T15:57:31,559 creating build/bdist.linux-armv7l/wheel/colossalai/amp 2023-09-06T15:57:31,561 copying build/lib/colossalai/amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-09-06T15:57:31,566 creating build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-09-06T15:57:31,567 copying build/lib/colossalai/amp/torch_amp/torch_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-09-06T15:57:31,571 copying build/lib/colossalai/amp/torch_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-09-06T15:57:31,574 copying build/lib/colossalai/amp/torch_amp/_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-09-06T15:57:31,580 copying build/lib/colossalai/amp/amp_type.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-09-06T15:57:31,584 creating build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-09-06T15:57:31,586 copying build/lib/colossalai/amp/apex_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-09-06T15:57:31,590 copying build/lib/colossalai/amp/apex_amp/apex_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-09-06T15:57:31,595 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-09-06T15:57:31,597 copying build/lib/colossalai/amp/naive_amp/mixed_precision_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-09-06T15:57:31,601 copying build/lib/colossalai/amp/naive_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-09-06T15:57:31,606 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:31,607 copying build/lib/colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:31,611 copying build/lib/colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:31,614 copying build/lib/colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:31,618 copying build/lib/colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-09-06T15:57:31,622 copying build/lib/colossalai/amp/naive_amp/naive_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-09-06T15:57:31,626 copying build/lib/colossalai/amp/naive_amp/_fp16_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-09-06T15:57:31,632 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:31,633 copying build/lib/colossalai/amp/naive_amp/mixed_precision_mixin/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:31,637 copying build/lib/colossalai/amp/naive_amp/mixed_precision_mixin/bf16.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:31,640 copying build/lib/colossalai/amp/naive_amp/mixed_precision_mixin/base.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:31,644 copying build/lib/colossalai/amp/naive_amp/mixed_precision_mixin/fp16.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/mixed_precision_mixin 2023-09-06T15:57:31,647 copying build/lib/colossalai/amp/naive_amp/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-09-06T15:57:31,652 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-09-06T15:57:31,654 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-09-06T15:57:31,657 copying build/lib/colossalai/auto_parallel/pipeline_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-09-06T15:57:31,660 copying build/lib/colossalai/auto_parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-09-06T15:57:31,664 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,665 copying build/lib/colossalai/auto_parallel/offload/amp_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,670 copying build/lib/colossalai/auto_parallel/offload/util.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,674 copying build/lib/colossalai/auto_parallel/offload/solver.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,679 copying build/lib/colossalai/auto_parallel/offload/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,682 copying build/lib/colossalai/auto_parallel/offload/region_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,687 copying build/lib/colossalai/auto_parallel/offload/mem_optimize.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,691 copying build/lib/colossalai/auto_parallel/offload/region.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,696 copying build/lib/colossalai/auto_parallel/offload/runtime.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,701 copying build/lib/colossalai/auto_parallel/offload/base_offload_module.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,706 copying build/lib/colossalai/auto_parallel/offload/training_simulator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/offload 2023-09-06T15:57:31,712 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-09-06T15:57:31,715 copying build/lib/colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-09-06T15:57:31,720 copying build/lib/colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-09-06T15:57:31,726 copying build/lib/colossalai/auto_parallel/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-09-06T15:57:31,731 copying build/lib/colossalai/auto_parallel/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-09-06T15:57:31,734 copying build/lib/colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-09-06T15:57:31,738 copying build/lib/colossalai/auto_parallel/passes/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-09-06T15:57:31,743 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:31,746 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,748 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,753 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,757 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,762 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,766 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,770 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,775 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,778 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,782 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,787 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,791 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,793 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,798 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,803 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,808 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,812 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,817 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,821 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,827 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,832 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,839 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,843 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,848 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,852 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,856 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,861 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,866 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,869 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,874 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,877 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-09-06T15:57:31,882 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,886 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,889 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,893 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,897 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,901 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,905 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,908 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,911 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,915 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,919 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,923 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,926 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,930 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,933 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,938 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,941 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-09-06T15:57:31,945 copying build/lib/colossalai/auto_parallel/tensor_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:31,948 copying build/lib/colossalai/auto_parallel/tensor_shard/options.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:31,952 copying build/lib/colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:31,957 copying build/lib/colossalai/auto_parallel/tensor_shard/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:31,963 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:31,965 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:31,971 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:31,974 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:31,979 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:31,983 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-09-06T15:57:31,988 copying build/lib/colossalai/auto_parallel/tensor_shard/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-09-06T15:57:31,993 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:31,994 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:31,999 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:32,003 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:32,008 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:32,013 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:32,018 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-09-06T15:57:32,024 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:32,026 copying build/lib/colossalai/auto_parallel/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:32,031 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:32,037 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:32,041 copying build/lib/colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:32,045 copying build/lib/colossalai/auto_parallel/checkpoint/operation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:32,049 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-09-06T15:57:32,055 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:32,058 copying build/lib/colossalai/auto_parallel/meta_profiler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:32,064 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,066 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,071 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,075 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,080 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,087 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,092 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,096 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,101 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,106 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/where.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,111 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,116 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-09-06T15:57:32,120 copying build/lib/colossalai/auto_parallel/meta_profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:32,124 copying build/lib/colossalai/auto_parallel/meta_profiler/shard_metainfo.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:32,129 copying build/lib/colossalai/auto_parallel/meta_profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-09-06T15:57:32,133 copying build/lib/colossalai/version.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-09-06T15:57:32,138 copying build/lib/colossalai/core.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-09-06T15:57:32,143 creating build/bdist.linux-armv7l/wheel/colossalai/_C 2023-09-06T15:57:32,145 copying build/lib/colossalai/_C/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_C 2023-09-06T15:57:32,150 creating build/bdist.linux-armv7l/wheel/colossalai/checkpoint_io 2023-09-06T15:57:32,153 copying build/lib/colossalai/checkpoint_io/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/checkpoint_io 2023-09-06T15:57:32,157 copying build/lib/colossalai/checkpoint_io/general_checkpoint_io.py -> build/bdist.linux-armv7l/wheel/colossalai/checkpoint_io 2023-09-06T15:57:32,162 copying build/lib/colossalai/checkpoint_io/checkpoint_io_base.py -> build/bdist.linux-armv7l/wheel/colossalai/checkpoint_io 2023-09-06T15:57:32,168 copying build/lib/colossalai/checkpoint_io/hybrid_parallel_checkpoint_io.py -> build/bdist.linux-armv7l/wheel/colossalai/checkpoint_io 2023-09-06T15:57:32,174 copying build/lib/colossalai/checkpoint_io/index_file.py -> build/bdist.linux-armv7l/wheel/colossalai/checkpoint_io 2023-09-06T15:57:32,179 copying build/lib/colossalai/checkpoint_io/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/checkpoint_io 2023-09-06T15:57:32,187 creating build/bdist.linux-armv7l/wheel/colossalai/cli 2023-09-06T15:57:32,189 copying build/lib/colossalai/cli/cli.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-09-06T15:57:32,195 creating build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-09-06T15:57:32,197 copying build/lib/colossalai/cli/benchmark/models.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-09-06T15:57:32,201 copying build/lib/colossalai/cli/benchmark/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-09-06T15:57:32,206 copying build/lib/colossalai/cli/benchmark/benchmark.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-09-06T15:57:32,211 copying build/lib/colossalai/cli/benchmark/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-09-06T15:57:32,216 copying build/lib/colossalai/cli/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-09-06T15:57:32,221 creating build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-09-06T15:57:32,224 copying build/lib/colossalai/cli/check/check_installation.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-09-06T15:57:32,229 copying build/lib/colossalai/cli/check/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-09-06T15:57:32,234 creating build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-09-06T15:57:32,237 copying build/lib/colossalai/cli/launcher/hostinfo.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-09-06T15:57:32,241 copying build/lib/colossalai/cli/launcher/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-09-06T15:57:32,246 copying build/lib/colossalai/cli/launcher/run.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-09-06T15:57:32,251 copying build/lib/colossalai/cli/launcher/multinode_runner.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-09-06T15:57:32,257 creating build/bdist.linux-armv7l/wheel/colossalai/fx 2023-09-06T15:57:32,259 copying build/lib/colossalai/fx/_meta_regist_12.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-09-06T15:57:32,265 copying build/lib/colossalai/fx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-09-06T15:57:32,269 copying build/lib/colossalai/fx/_meta_regist_13.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-09-06T15:57:32,275 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,278 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-09-06T15:57:32,281 copying build/lib/colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-09-06T15:57:32,286 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:32,289 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:32,293 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:32,298 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:32,302 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:32,307 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-09-06T15:57:32,312 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:32,314 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:32,319 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:32,323 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:32,328 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-09-06T15:57:32,332 copying build/lib/colossalai/fx/tracer/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,336 copying build/lib/colossalai/fx/tracer/experimental.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,343 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-09-06T15:57:32,347 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,349 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,354 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,358 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,362 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,367 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,372 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,376 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,381 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-09-06T15:57:32,387 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,389 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,393 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,398 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,402 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,406 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,411 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,415 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,419 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-09-06T15:57:32,424 copying build/lib/colossalai/fx/tracer/meta_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-09-06T15:57:32,428 copying build/lib/colossalai/fx/tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,432 copying build/lib/colossalai/fx/tracer/_symbolic_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,436 copying build/lib/colossalai/fx/tracer/_meta_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,441 copying build/lib/colossalai/fx/tracer/_tracer_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,445 copying build/lib/colossalai/fx/tracer/tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-09-06T15:57:32,452 creating build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-09-06T15:57:32,454 copying build/lib/colossalai/fx/codegen/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-09-06T15:57:32,458 copying build/lib/colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-09-06T15:57:32,466 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,469 copying build/lib/colossalai/fx/profiler/opcount.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,474 copying build/lib/colossalai/fx/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,480 copying build/lib/colossalai/fx/profiler/dataflow.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,484 copying build/lib/colossalai/fx/profiler/memory_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,489 copying build/lib/colossalai/fx/profiler/tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,493 copying build/lib/colossalai/fx/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,498 copying build/lib/colossalai/fx/profiler/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,504 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-09-06T15:57:32,508 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,511 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,515 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,519 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,523 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,526 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,530 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,533 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,538 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,541 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,545 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,549 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-09-06T15:57:32,552 copying build/lib/colossalai/fx/profiler/experimental/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-09-06T15:57:32,557 copying build/lib/colossalai/fx/profiler/experimental/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-09-06T15:57:32,560 copying build/lib/colossalai/fx/profiler/experimental/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-09-06T15:57:32,564 copying build/lib/colossalai/fx/profiler/experimental/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-09-06T15:57:32,568 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,570 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,574 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,578 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,581 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,585 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,588 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,592 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,596 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,600 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-09-06T15:57:32,603 copying build/lib/colossalai/fx/profiler/experimental/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-09-06T15:57:32,607 copying build/lib/colossalai/fx/profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-09-06T15:57:32,611 copying build/lib/colossalai/fx/graph_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-09-06T15:57:32,615 copying build/lib/colossalai/fx/proxy.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-09-06T15:57:32,621 creating build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,622 copying build/lib/colossalai/fx/passes/shard_1d_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,627 copying build/lib/colossalai/fx/passes/adding_split_node_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,632 copying build/lib/colossalai/fx/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,636 copying build/lib/colossalai/fx/passes/passes_for_gpt2_test.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,641 copying build/lib/colossalai/fx/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,644 copying build/lib/colossalai/fx/passes/concrete_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,648 copying build/lib/colossalai/fx/passes/split_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,653 copying build/lib/colossalai/fx/passes/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-09-06T15:57:32,658 copying build/lib/colossalai/fx/_compatibility.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-09-06T15:57:32,661 copying build/lib/colossalai/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-09-06T15:57:32,666 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,668 copying build/lib/colossalai/pipeline/pipelinable.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,673 copying build/lib/colossalai/pipeline/stage_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,677 copying build/lib/colossalai/pipeline/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,682 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-09-06T15:57:32,684 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-09-06T15:57:32,687 copying build/lib/colossalai/pipeline/middleware/adaptor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-09-06T15:57:32,690 copying build/lib/colossalai/pipeline/middleware/adaptor/fx.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-09-06T15:57:32,695 copying build/lib/colossalai/pipeline/middleware/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-09-06T15:57:32,699 copying build/lib/colossalai/pipeline/middleware/topo.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-09-06T15:57:32,703 copying build/lib/colossalai/pipeline/layer_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,708 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/schedule 2023-09-06T15:57:32,710 copying build/lib/colossalai/pipeline/schedule/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/schedule 2023-09-06T15:57:32,714 copying build/lib/colossalai/pipeline/schedule/one_f_one_b.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/schedule 2023-09-06T15:57:32,719 copying build/lib/colossalai/pipeline/schedule/base.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/schedule 2023-09-06T15:57:32,722 copying build/lib/colossalai/pipeline/schedule/interleaved_pp.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/schedule 2023-09-06T15:57:32,727 copying build/lib/colossalai/pipeline/schedule/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/schedule 2023-09-06T15:57:32,731 copying build/lib/colossalai/pipeline/pipeline_process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,736 copying build/lib/colossalai/pipeline/p2p.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,741 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-09-06T15:57:32,743 copying build/lib/colossalai/pipeline/rpc/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-09-06T15:57:32,750 copying build/lib/colossalai/pipeline/rpc/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-09-06T15:57:32,753 copying build/lib/colossalai/pipeline/rpc/_pipeline_base.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-09-06T15:57:32,760 copying build/lib/colossalai/pipeline/rpc/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-09-06T15:57:32,764 copying build/lib/colossalai/pipeline/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-09-06T15:57:32,770 creating build/bdist.linux-armv7l/wheel/colossalai/lazy 2023-09-06T15:57:32,771 copying build/lib/colossalai/lazy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/lazy 2023-09-06T15:57:32,775 copying build/lib/colossalai/lazy/lazy_init.py -> build/bdist.linux-armv7l/wheel/colossalai/lazy 2023-09-06T15:57:32,782 creating build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,784 copying build/lib/colossalai/tensor/shape_consistency.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,790 copying build/lib/colossalai/tensor/tensor_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,794 copying build/lib/colossalai/tensor/sharding_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,799 copying build/lib/colossalai/tensor/dist_spec_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,803 copying build/lib/colossalai/tensor/op_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,808 creating build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,810 copying build/lib/colossalai/tensor/d_tensor/sharding_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,815 copying build/lib/colossalai/tensor/d_tensor/layout.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,818 copying build/lib/colossalai/tensor/d_tensor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,822 copying build/lib/colossalai/tensor/d_tensor/layout_converter.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,828 copying build/lib/colossalai/tensor/d_tensor/api.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,832 copying build/lib/colossalai/tensor/d_tensor/misc.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,836 copying build/lib/colossalai/tensor/d_tensor/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,840 copying build/lib/colossalai/tensor/d_tensor/comm_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-09-06T15:57:32,844 copying build/lib/colossalai/tensor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,848 copying build/lib/colossalai/tensor/const.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,852 copying build/lib/colossalai/tensor/process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,856 copying build/lib/colossalai/tensor/colo_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,860 copying build/lib/colossalai/tensor/compute_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,864 copying build/lib/colossalai/tensor/colo_parameter.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,867 copying build/lib/colossalai/tensor/param_op_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,872 copying build/lib/colossalai/tensor/distspec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,875 copying build/lib/colossalai/tensor/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,879 copying build/lib/colossalai/tensor/comm_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-09-06T15:57:32,886 creating build/bdist.linux-armv7l/wheel/colossalai/_analyzer 2023-09-06T15:57:32,889 creating build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx 2023-09-06T15:57:32,890 copying build/lib/colossalai/_analyzer/fx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx 2023-09-06T15:57:32,895 creating build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:32,897 copying build/lib/colossalai/_analyzer/fx/tracer/symbolic_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:32,901 copying build/lib/colossalai/_analyzer/fx/tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:32,905 copying build/lib/colossalai/_analyzer/fx/tracer/bias_addition.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:32,909 copying build/lib/colossalai/_analyzer/fx/tracer/proxy.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:32,913 copying build/lib/colossalai/_analyzer/fx/tracer/custom_leaf_module.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:32,916 copying build/lib/colossalai/_analyzer/fx/tracer/tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/tracer 2023-09-06T15:57:32,921 copying build/lib/colossalai/_analyzer/fx/node_util.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx 2023-09-06T15:57:32,926 copying build/lib/colossalai/_analyzer/fx/codegen.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx 2023-09-06T15:57:32,931 copying build/lib/colossalai/_analyzer/fx/symbolic_profile.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx 2023-09-06T15:57:32,934 copying build/lib/colossalai/_analyzer/fx/graph_module.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx 2023-09-06T15:57:32,940 creating build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/passes 2023-09-06T15:57:32,941 copying build/lib/colossalai/_analyzer/fx/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/passes 2023-09-06T15:57:32,945 copying build/lib/colossalai/_analyzer/fx/passes/shape_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/passes 2023-09-06T15:57:32,950 copying build/lib/colossalai/_analyzer/fx/passes/graph_profile.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/fx/passes 2023-09-06T15:57:32,954 copying build/lib/colossalai/_analyzer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer 2023-09-06T15:57:32,959 creating build/bdist.linux-armv7l/wheel/colossalai/_analyzer/_subclasses 2023-09-06T15:57:32,960 copying build/lib/colossalai/_analyzer/_subclasses/_meta_registration.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/_subclasses 2023-09-06T15:57:32,966 copying build/lib/colossalai/_analyzer/_subclasses/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/_subclasses 2023-09-06T15:57:32,969 copying build/lib/colossalai/_analyzer/_subclasses/flop_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/_subclasses 2023-09-06T15:57:32,974 copying build/lib/colossalai/_analyzer/_subclasses/meta_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/_subclasses 2023-09-06T15:57:32,978 copying build/lib/colossalai/_analyzer/_subclasses/_monkey_patch.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer/_subclasses 2023-09-06T15:57:32,982 copying build/lib/colossalai/_analyzer/envs.py -> build/bdist.linux-armv7l/wheel/colossalai/_analyzer 2023-09-06T15:57:32,985 copying build/lib/colossalai/global_variables.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-09-06T15:57:32,990 creating build/bdist.linux-armv7l/wheel/colossalai/interface 2023-09-06T15:57:32,992 copying build/lib/colossalai/interface/model.py -> build/bdist.linux-armv7l/wheel/colossalai/interface 2023-09-06T15:57:32,995 copying build/lib/colossalai/interface/optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/interface 2023-09-06T15:57:32,999 copying build/lib/colossalai/interface/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/interface 2023-09-06T15:57:33,003 creating build/bdist.linux-armv7l/wheel/colossalai/booster 2023-09-06T15:57:33,007 creating build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,009 copying build/lib/colossalai/booster/plugin/gemini_plugin.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,014 copying build/lib/colossalai/booster/plugin/low_level_zero_plugin.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,019 copying build/lib/colossalai/booster/plugin/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,022 copying build/lib/colossalai/booster/plugin/dp_plugin_base.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,026 copying build/lib/colossalai/booster/plugin/torch_fsdp_plugin.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,031 copying build/lib/colossalai/booster/plugin/pp_plugin_base.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,034 copying build/lib/colossalai/booster/plugin/torch_ddp_plugin.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,038 copying build/lib/colossalai/booster/plugin/plugin_base.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,041 copying build/lib/colossalai/booster/plugin/hybrid_parallel_plugin.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/plugin 2023-09-06T15:57:33,047 copying build/lib/colossalai/booster/booster.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-09-06T15:57:33,052 copying build/lib/colossalai/booster/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-09-06T15:57:33,056 creating build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,058 copying build/lib/colossalai/booster/mixed_precision/fp8.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,062 copying build/lib/colossalai/booster/mixed_precision/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,066 copying build/lib/colossalai/booster/mixed_precision/bf16.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,069 copying build/lib/colossalai/booster/mixed_precision/fp16_naive.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,073 copying build/lib/colossalai/booster/mixed_precision/fp16_torch.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,077 copying build/lib/colossalai/booster/mixed_precision/mixed_precision_base.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,081 copying build/lib/colossalai/booster/mixed_precision/fp16_apex.py -> build/bdist.linux-armv7l/wheel/colossalai/booster/mixed_precision 2023-09-06T15:57:33,084 copying build/lib/colossalai/booster/accelerator.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-09-06T15:57:33,090 creating build/bdist.linux-armv7l/wheel/colossalai/cluster 2023-09-06T15:57:33,091 copying build/lib/colossalai/cluster/device_mesh_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/cluster 2023-09-06T15:57:33,096 copying build/lib/colossalai/cluster/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cluster 2023-09-06T15:57:33,099 copying build/lib/colossalai/cluster/dist_coordinator.py -> build/bdist.linux-armv7l/wheel/colossalai/cluster 2023-09-06T15:57:33,103 copying build/lib/colossalai/cluster/process_group_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/cluster 2023-09-06T15:57:33,107 copying build/lib/colossalai/cluster/process_group_mesh.py -> build/bdist.linux-armv7l/wheel/colossalai/cluster 2023-09-06T15:57:33,112 creating build/bdist.linux-armv7l/wheel/colossalai/device 2023-09-06T15:57:33,114 copying build/lib/colossalai/device/calc_pipeline_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-09-06T15:57:33,119 copying build/lib/colossalai/device/alpha_beta_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-09-06T15:57:33,124 copying build/lib/colossalai/device/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-09-06T15:57:33,127 copying build/lib/colossalai/device/device_mesh.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-09-06T15:57:33,133 creating build/bdist.linux-armv7l/wheel/colossalai/legacy 2023-09-06T15:57:33,136 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/builder 2023-09-06T15:57:33,138 copying build/lib/colossalai/legacy/builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/builder 2023-09-06T15:57:33,142 copying build/lib/colossalai/legacy/builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/builder 2023-09-06T15:57:33,146 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/registry 2023-09-06T15:57:33,148 copying build/lib/colossalai/legacy/registry/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/registry 2023-09-06T15:57:33,152 copying build/lib/colossalai/legacy/registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/registry 2023-09-06T15:57:33,156 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer 2023-09-06T15:57:33,158 copying build/lib/colossalai/legacy/trainer/_trainer.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer 2023-09-06T15:57:33,162 copying build/lib/colossalai/legacy/trainer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer 2023-09-06T15:57:33,167 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,169 copying build/lib/colossalai/legacy/trainer/hooks/_base_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,173 copying build/lib/colossalai/legacy/trainer/hooks/_log_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,177 copying build/lib/colossalai/legacy/trainer/hooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,181 copying build/lib/colossalai/legacy/trainer/hooks/_metric_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,185 copying build/lib/colossalai/legacy/trainer/hooks/_commons_.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,188 copying build/lib/colossalai/legacy/trainer/hooks/_lr_scheduler_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,192 copying build/lib/colossalai/legacy/trainer/hooks/_checkpoint_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/trainer/hooks 2023-09-06T15:57:33,195 copying build/lib/colossalai/legacy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy 2023-09-06T15:57:33,199 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/engine 2023-09-06T15:57:33,201 copying build/lib/colossalai/legacy/engine/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine 2023-09-06T15:57:33,205 copying build/lib/colossalai/legacy/engine/_base_engine.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine 2023-09-06T15:57:33,210 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/schedule 2023-09-06T15:57:33,211 copying build/lib/colossalai/legacy/engine/schedule/_base_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/schedule 2023-09-06T15:57:33,216 copying build/lib/colossalai/legacy/engine/schedule/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/schedule 2023-09-06T15:57:33,221 copying build/lib/colossalai/legacy/engine/schedule/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/schedule 2023-09-06T15:57:33,225 copying build/lib/colossalai/legacy/engine/schedule/_pipeline_schedule_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/schedule 2023-09-06T15:57:33,229 copying build/lib/colossalai/legacy/engine/schedule/_non_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/schedule 2023-09-06T15:57:33,233 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_accumulation 2023-09-06T15:57:33,235 copying build/lib/colossalai/legacy/engine/gradient_accumulation/_gradient_accumulation.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_accumulation 2023-09-06T15:57:33,240 copying build/lib/colossalai/legacy/engine/gradient_accumulation/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_accumulation 2023-09-06T15:57:33,244 creating build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,246 copying build/lib/colossalai/legacy/engine/gradient_handler/_moe_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,250 copying build/lib/colossalai/legacy/engine/gradient_handler/_base_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,253 copying build/lib/colossalai/legacy/engine/gradient_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,257 copying build/lib/colossalai/legacy/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,260 copying build/lib/colossalai/legacy/engine/gradient_handler/_zero_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,263 copying build/lib/colossalai/legacy/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,267 copying build/lib/colossalai/legacy/engine/gradient_handler/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,270 copying build/lib/colossalai/legacy/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/legacy/engine/gradient_handler 2023-09-06T15:57:33,275 creating build/bdist.linux-armv7l/wheel/colossalai/zero 2023-09-06T15:57:33,276 copying build/lib/colossalai/zero/wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/zero 2023-09-06T15:57:33,281 copying build/lib/colossalai/zero/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero 2023-09-06T15:57:33,285 creating build/bdist.linux-armv7l/wheel/colossalai/zero/low_level 2023-09-06T15:57:33,287 copying build/lib/colossalai/zero/low_level/low_level_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level 2023-09-06T15:57:33,294 creating build/bdist.linux-armv7l/wheel/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:33,295 copying build/lib/colossalai/zero/low_level/bookkeeping/bucket_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:33,300 copying build/lib/colossalai/zero/low_level/bookkeeping/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:33,303 copying build/lib/colossalai/zero/low_level/bookkeeping/parameter_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:33,307 copying build/lib/colossalai/zero/low_level/bookkeeping/tensor_bucket.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:33,310 copying build/lib/colossalai/zero/low_level/bookkeeping/gradient_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:33,314 copying build/lib/colossalai/zero/low_level/bookkeeping/base_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level/bookkeeping 2023-09-06T15:57:33,317 copying build/lib/colossalai/zero/low_level/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level 2023-09-06T15:57:33,321 copying build/lib/colossalai/zero/low_level/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/low_level 2023-09-06T15:57:33,326 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy 2023-09-06T15:57:33,329 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:33,331 copying build/lib/colossalai/zero/legacy/sharded_model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:33,335 copying build/lib/colossalai/zero/legacy/sharded_model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:33,338 copying build/lib/colossalai/zero/legacy/sharded_model/reduce_scatter.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:33,342 copying build/lib/colossalai/zero/legacy/sharded_model/sharded_model_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:33,348 copying build/lib/colossalai/zero/legacy/sharded_model/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:33,351 copying build/lib/colossalai/zero/legacy/sharded_model/zero_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_model 2023-09-06T15:57:33,356 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:33,358 copying build/lib/colossalai/zero/legacy/shard_utils/base_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:33,362 copying build/lib/colossalai/zero/legacy/shard_utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:33,366 copying build/lib/colossalai/zero/legacy/shard_utils/bucket_tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:33,371 copying build/lib/colossalai/zero/legacy/shard_utils/commons.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:33,375 copying build/lib/colossalai/zero/legacy/shard_utils/tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/shard_utils 2023-09-06T15:57:33,380 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/init_ctx 2023-09-06T15:57:33,383 copying build/lib/colossalai/zero/legacy/init_ctx/init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/init_ctx 2023-09-06T15:57:33,388 copying build/lib/colossalai/zero/legacy/init_ctx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/init_ctx 2023-09-06T15:57:33,392 copying build/lib/colossalai/zero/legacy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy 2023-09-06T15:57:33,397 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini 2023-09-06T15:57:33,399 copying build/lib/colossalai/zero/legacy/gemini/tensor_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini 2023-09-06T15:57:33,403 copying build/lib/colossalai/zero/legacy/gemini/gemini_context.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini 2023-09-06T15:57:33,408 copying build/lib/colossalai/zero/legacy/gemini/stateful_tensor_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini 2023-09-06T15:57:33,412 copying build/lib/colossalai/zero/legacy/gemini/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini 2023-09-06T15:57:33,417 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/paramhooks 2023-09-06T15:57:33,420 copying build/lib/colossalai/zero/legacy/gemini/paramhooks/_param_hookmgr.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/paramhooks 2023-09-06T15:57:33,424 copying build/lib/colossalai/zero/legacy/gemini/paramhooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/paramhooks 2023-09-06T15:57:33,428 copying build/lib/colossalai/zero/legacy/gemini/stateful_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini 2023-09-06T15:57:33,432 copying build/lib/colossalai/zero/legacy/gemini/tensor_placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini 2023-09-06T15:57:33,438 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:33,441 copying build/lib/colossalai/zero/legacy/gemini/ophooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:33,445 copying build/lib/colossalai/zero/legacy/gemini/ophooks/runtime_mem_tracer_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:33,450 copying build/lib/colossalai/zero/legacy/gemini/ophooks/_shard_grad_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:33,454 copying build/lib/colossalai/zero/legacy/gemini/ophooks/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:33,459 copying build/lib/colossalai/zero/legacy/gemini/ophooks/_shard_param_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/gemini/ophooks 2023-09-06T15:57:33,464 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:33,466 copying build/lib/colossalai/zero/legacy/sharded_param/sharded_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:33,470 copying build/lib/colossalai/zero/legacy/sharded_param/sharded_param.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:33,474 copying build/lib/colossalai/zero/legacy/sharded_param/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_param 2023-09-06T15:57:33,480 creating build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_optim 2023-09-06T15:57:33,482 copying build/lib/colossalai/zero/legacy/sharded_optim/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_optim 2023-09-06T15:57:33,486 copying build/lib/colossalai/zero/legacy/sharded_optim/sharded_optim_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/legacy/sharded_optim 2023-09-06T15:57:33,493 creating build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,496 copying build/lib/colossalai/zero/gemini/gemini_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,502 copying build/lib/colossalai/zero/gemini/placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,507 copying build/lib/colossalai/zero/gemini/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,513 creating build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,515 copying build/lib/colossalai/zero/gemini/memory_tracer/memory_monitor.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,519 copying build/lib/colossalai/zero/gemini/memory_tracer/param_runtime_order.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,523 copying build/lib/colossalai/zero/gemini/memory_tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,526 copying build/lib/colossalai/zero/gemini/memory_tracer/runtime_mem_tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,530 copying build/lib/colossalai/zero/gemini/memory_tracer/memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,534 copying build/lib/colossalai/zero/gemini/memory_tracer/static_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,538 copying build/lib/colossalai/zero/gemini/memory_tracer/memory_stats.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,542 copying build/lib/colossalai/zero/gemini/memory_tracer/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,545 copying build/lib/colossalai/zero/gemini/memory_tracer/chunk_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/memory_tracer 2023-09-06T15:57:33,549 copying build/lib/colossalai/zero/gemini/colo_init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,553 copying build/lib/colossalai/zero/gemini/gemini_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,556 copying build/lib/colossalai/zero/gemini/gemini_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,560 copying build/lib/colossalai/zero/gemini/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,565 creating build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/chunk 2023-09-06T15:57:33,567 copying build/lib/colossalai/zero/gemini/chunk/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/chunk 2023-09-06T15:57:33,571 copying build/lib/colossalai/zero/gemini/chunk/chunk.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/chunk 2023-09-06T15:57:33,576 copying build/lib/colossalai/zero/gemini/chunk/manager.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/chunk 2023-09-06T15:57:33,580 copying build/lib/colossalai/zero/gemini/chunk/search_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/chunk 2023-09-06T15:57:33,584 copying build/lib/colossalai/zero/gemini/chunk/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini/chunk 2023-09-06T15:57:33,588 copying build/lib/colossalai/zero/gemini/gemini_ddp.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/gemini 2023-09-06T15:57:33,595 creating build/bdist.linux-armv7l/wheel/colossalai/testing 2023-09-06T15:57:33,597 copying build/lib/colossalai/testing/pytest_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-09-06T15:57:33,601 copying build/lib/colossalai/testing/random.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-09-06T15:57:33,604 copying build/lib/colossalai/testing/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-09-06T15:57:33,608 copying build/lib/colossalai/testing/comparison.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-09-06T15:57:33,612 copying build/lib/colossalai/testing/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-09-06T15:57:33,617 copying build/lib/colossalai/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-09-06T15:57:33,623 creating build/bdist.linux-armv7l/wheel/colossalai/nn 2023-09-06T15:57:33,626 creating build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-09-06T15:57:33,628 copying build/lib/colossalai/nn/loss/loss_moe.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-09-06T15:57:33,631 copying build/lib/colossalai/nn/loss/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-09-06T15:57:33,635 copying build/lib/colossalai/nn/loss/loss_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-09-06T15:57:33,639 copying build/lib/colossalai/nn/loss/loss_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-09-06T15:57:33,643 copying build/lib/colossalai/nn/loss/loss_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-09-06T15:57:33,647 copying build/lib/colossalai/nn/loss/loss_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-09-06T15:57:33,651 copying build/lib/colossalai/nn/init.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-09-06T15:57:33,655 copying build/lib/colossalai/nn/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-09-06T15:57:33,660 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-09-06T15:57:33,662 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-09-06T15:57:33,664 copying build/lib/colossalai/nn/layer/vanilla/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-09-06T15:57:33,668 copying build/lib/colossalai/nn/layer/vanilla/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-09-06T15:57:33,674 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:33,676 copying build/lib/colossalai/nn/layer/parallel_sequence/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:33,679 copying build/lib/colossalai/nn/layer/parallel_sequence/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:33,683 copying build/lib/colossalai/nn/layer/parallel_sequence/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:33,688 copying build/lib/colossalai/nn/layer/parallel_sequence/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-09-06T15:57:33,692 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:33,694 copying build/lib/colossalai/nn/layer/colossalai_layer/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:33,697 copying build/lib/colossalai/nn/layer/colossalai_layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:33,701 copying build/lib/colossalai/nn/layer/colossalai_layer/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:33,705 copying build/lib/colossalai/nn/layer/colossalai_layer/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:33,708 copying build/lib/colossalai/nn/layer/colossalai_layer/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:33,712 copying build/lib/colossalai/nn/layer/colossalai_layer/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-09-06T15:57:33,717 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:33,718 copying build/lib/colossalai/nn/layer/parallel_2p5d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:33,722 copying build/lib/colossalai/nn/layer/parallel_2p5d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:33,728 copying build/lib/colossalai/nn/layer/parallel_2p5d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:33,734 copying build/lib/colossalai/nn/layer/parallel_2p5d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-09-06T15:57:33,739 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:33,740 copying build/lib/colossalai/nn/layer/parallel_3d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:33,744 copying build/lib/colossalai/nn/layer/parallel_3d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:33,749 copying build/lib/colossalai/nn/layer/parallel_3d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:33,755 copying build/lib/colossalai/nn/layer/parallel_3d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-09-06T15:57:33,759 copying build/lib/colossalai/nn/layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-09-06T15:57:33,763 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:33,765 copying build/lib/colossalai/nn/layer/parallel_2d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:33,768 copying build/lib/colossalai/nn/layer/parallel_2d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:33,774 copying build/lib/colossalai/nn/layer/parallel_2d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:33,780 copying build/lib/colossalai/nn/layer/parallel_2d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-09-06T15:57:33,784 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:33,786 copying build/lib/colossalai/nn/layer/parallel_1d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:33,790 copying build/lib/colossalai/nn/layer/parallel_1d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:33,793 copying build/lib/colossalai/nn/layer/parallel_1d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:33,800 copying build/lib/colossalai/nn/layer/parallel_1d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-09-06T15:57:33,805 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,807 copying build/lib/colossalai/nn/layer/moe/experts.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,811 copying build/lib/colossalai/nn/layer/moe/checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,815 copying build/lib/colossalai/nn/layer/moe/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,818 copying build/lib/colossalai/nn/layer/moe/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,822 copying build/lib/colossalai/nn/layer/moe/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,827 copying build/lib/colossalai/nn/layer/moe/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,831 copying build/lib/colossalai/nn/layer/moe/routers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-09-06T15:57:33,835 copying build/lib/colossalai/nn/layer/base_layer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-09-06T15:57:33,839 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-09-06T15:57:33,841 copying build/lib/colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-09-06T15:57:33,845 copying build/lib/colossalai/nn/layer/wrapper/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-09-06T15:57:33,849 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-09-06T15:57:33,851 copying build/lib/colossalai/nn/layer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-09-06T15:57:33,854 copying build/lib/colossalai/nn/layer/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-09-06T15:57:33,859 creating build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-09-06T15:57:33,861 copying build/lib/colossalai/nn/metric/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-09-06T15:57:33,865 copying build/lib/colossalai/nn/metric/accuracy_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-09-06T15:57:33,868 copying build/lib/colossalai/nn/metric/accuracy_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-09-06T15:57:33,872 copying build/lib/colossalai/nn/metric/accuracy_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-09-06T15:57:33,875 copying build/lib/colossalai/nn/metric/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-09-06T15:57:33,880 creating build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,881 copying build/lib/colossalai/nn/lr_scheduler/torch.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,885 copying build/lib/colossalai/nn/lr_scheduler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,889 copying build/lib/colossalai/nn/lr_scheduler/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,892 copying build/lib/colossalai/nn/lr_scheduler/onecycle.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,896 copying build/lib/colossalai/nn/lr_scheduler/cosine.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,901 copying build/lib/colossalai/nn/lr_scheduler/delayed.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,905 copying build/lib/colossalai/nn/lr_scheduler/poly.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,909 copying build/lib/colossalai/nn/lr_scheduler/multistep.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-09-06T15:57:33,914 creating build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,916 copying build/lib/colossalai/nn/_ops/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,920 copying build/lib/colossalai/nn/_ops/batch_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,923 copying build/lib/colossalai/nn/_ops/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,927 copying build/lib/colossalai/nn/_ops/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,932 copying build/lib/colossalai/nn/_ops/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,935 copying build/lib/colossalai/nn/_ops/loss.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,939 copying build/lib/colossalai/nn/_ops/embedding_bag.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,943 copying build/lib/colossalai/nn/_ops/element_wise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,948 copying build/lib/colossalai/nn/_ops/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,953 copying build/lib/colossalai/nn/_ops/view.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,957 copying build/lib/colossalai/nn/_ops/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-09-06T15:57:33,962 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-09-06T15:57:33,964 copying build/lib/colossalai/nn/parallel/data_parallel.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-09-06T15:57:33,969 copying build/lib/colossalai/nn/parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-09-06T15:57:33,972 copying build/lib/colossalai/nn/parallel/reducer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-09-06T15:57:33,977 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-09-06T15:57:33,979 copying build/lib/colossalai/nn/parallel/layers/colo_module.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-09-06T15:57:33,983 copying build/lib/colossalai/nn/parallel/layers/module_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-09-06T15:57:33,988 copying build/lib/colossalai/nn/parallel/layers/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-09-06T15:57:33,992 copying build/lib/colossalai/nn/parallel/layers/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-09-06T15:57:33,996 copying build/lib/colossalai/nn/parallel/layers/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-09-06T15:57:34,002 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,005 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,010 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,015 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,019 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,024 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,029 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,034 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,039 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,044 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-09-06T15:57:34,051 creating build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,054 copying build/lib/colossalai/nn/optimizer/fused_lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,059 copying build/lib/colossalai/nn/optimizer/lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,064 copying build/lib/colossalai/nn/optimizer/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,069 copying build/lib/colossalai/nn/optimizer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,074 copying build/lib/colossalai/nn/optimizer/fused_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,078 copying build/lib/colossalai/nn/optimizer/hybrid_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,083 copying build/lib/colossalai/nn/optimizer/fused_sgd.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,088 copying build/lib/colossalai/nn/optimizer/lars.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,093 copying build/lib/colossalai/nn/optimizer/colossalai_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,097 copying build/lib/colossalai/nn/optimizer/nvme_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-09-06T15:57:34,103 creating build/bdist.linux-armv7l/wheel/colossalai/logging 2023-09-06T15:57:34,106 copying build/lib/colossalai/logging/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-09-06T15:57:34,110 copying build/lib/colossalai/logging/logger.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-09-06T15:57:34,116 creating build/bdist.linux-armv7l/wheel/colossalai/shardformer 2023-09-06T15:57:34,119 creating build/bdist.linux-armv7l/wheel/colossalai/shardformer/shard 2023-09-06T15:57:34,122 copying build/lib/colossalai/shardformer/shard/shard_config.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/shard 2023-09-06T15:57:34,126 copying build/lib/colossalai/shardformer/shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/shard 2023-09-06T15:57:34,131 copying build/lib/colossalai/shardformer/shard/shardformer.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/shard 2023-09-06T15:57:34,135 copying build/lib/colossalai/shardformer/shard/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/shard 2023-09-06T15:57:34,139 copying build/lib/colossalai/shardformer/shard/sharder.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/shard 2023-09-06T15:57:34,143 copying build/lib/colossalai/shardformer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer 2023-09-06T15:57:34,148 creating build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,150 copying build/lib/colossalai/shardformer/layer/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,154 copying build/lib/colossalai/shardformer/layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,157 copying build/lib/colossalai/shardformer/layer/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,162 copying build/lib/colossalai/shardformer/layer/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,167 copying build/lib/colossalai/shardformer/layer/loss.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,171 copying build/lib/colossalai/shardformer/layer/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,175 copying build/lib/colossalai/shardformer/layer/parallel_module.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,179 copying build/lib/colossalai/shardformer/layer/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,184 copying build/lib/colossalai/shardformer/layer/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,189 copying build/lib/colossalai/shardformer/layer/qkv_fused_linear.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/layer 2023-09-06T15:57:34,196 creating build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,197 copying build/lib/colossalai/shardformer/policies/llama.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,202 copying build/lib/colossalai/shardformer/policies/vit.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,207 copying build/lib/colossalai/shardformer/policies/t5.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,212 copying build/lib/colossalai/shardformer/policies/base_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,217 copying build/lib/colossalai/shardformer/policies/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,220 copying build/lib/colossalai/shardformer/policies/sam.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,225 copying build/lib/colossalai/shardformer/policies/gpt2.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,230 copying build/lib/colossalai/shardformer/policies/opt.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,234 copying build/lib/colossalai/shardformer/policies/bert.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,240 copying build/lib/colossalai/shardformer/policies/chatglm2.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,246 copying build/lib/colossalai/shardformer/policies/whisper.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,252 copying build/lib/colossalai/shardformer/policies/auto_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,256 copying build/lib/colossalai/shardformer/policies/blip2.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,261 copying build/lib/colossalai/shardformer/policies/bloom.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/policies 2023-09-06T15:57:34,267 creating build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,269 copying build/lib/colossalai/shardformer/modeling/llama.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,274 copying build/lib/colossalai/shardformer/modeling/vit.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,279 copying build/lib/colossalai/shardformer/modeling/t5.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,284 copying build/lib/colossalai/shardformer/modeling/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,287 copying build/lib/colossalai/shardformer/modeling/sam.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,292 copying build/lib/colossalai/shardformer/modeling/gpt2.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,298 copying build/lib/colossalai/shardformer/modeling/jit.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,301 copying build/lib/colossalai/shardformer/modeling/opt.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,307 copying build/lib/colossalai/shardformer/modeling/bert.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,314 copying build/lib/colossalai/shardformer/modeling/chatglm2.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,319 copying build/lib/colossalai/shardformer/modeling/whisper.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,326 copying build/lib/colossalai/shardformer/modeling/blip2.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,330 copying build/lib/colossalai/shardformer/modeling/bloom.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer/modeling 2023-09-06T15:57:34,337 copying build/lib/colossalai/shardformer/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/shardformer 2023-09-06T15:57:34,342 creating build/bdist.linux-armv7l/wheel/colossalai/communication 2023-09-06T15:57:34,344 copying build/lib/colossalai/communication/p2p_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-09-06T15:57:34,349 copying build/lib/colossalai/communication/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-09-06T15:57:34,352 copying build/lib/colossalai/communication/collective.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-09-06T15:57:34,357 copying build/lib/colossalai/communication/ring.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-09-06T15:57:34,360 copying build/lib/colossalai/communication/p2p.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-09-06T15:57:34,365 copying build/lib/colossalai/communication/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-09-06T15:57:34,370 copying build/lib/colossalai/constants.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-09-06T15:57:34,374 creating build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,376 copying build/lib/colossalai/utils/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,381 creating build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-09-06T15:57:34,383 copying build/lib/colossalai/utils/data_sampler/data_parallel_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-09-06T15:57:34,387 copying build/lib/colossalai/utils/data_sampler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-09-06T15:57:34,391 copying build/lib/colossalai/utils/data_sampler/base_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-09-06T15:57:34,394 copying build/lib/colossalai/utils/memory.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,400 creating build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-09-06T15:57:34,401 copying build/lib/colossalai/utils/tensor_detector/tensor_detector.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-09-06T15:57:34,406 copying build/lib/colossalai/utils/tensor_detector/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-09-06T15:57:34,410 copying build/lib/colossalai/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,413 copying build/lib/colossalai/utils/cuda.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,417 copying build/lib/colossalai/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,422 copying build/lib/colossalai/utils/checkpointing.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,427 copying build/lib/colossalai/utils/activation_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,433 creating build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-09-06T15:57:34,434 copying build/lib/colossalai/utils/model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-09-06T15:57:34,438 copying build/lib/colossalai/utils/model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-09-06T15:57:34,443 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-09-06T15:57:34,444 copying build/lib/colossalai/utils/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-09-06T15:57:34,449 copying build/lib/colossalai/utils/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-09-06T15:57:34,454 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-09-06T15:57:34,456 copying build/lib/colossalai/utils/profiler/legacy/prof_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-09-06T15:57:34,460 copying build/lib/colossalai/utils/profiler/legacy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-09-06T15:57:34,463 copying build/lib/colossalai/utils/profiler/legacy/comm_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-09-06T15:57:34,468 copying build/lib/colossalai/utils/profiler/legacy/pcie_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-09-06T15:57:34,473 copying build/lib/colossalai/utils/profiler/extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-09-06T15:57:34,477 copying build/lib/colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-09-06T15:57:34,483 creating build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-09-06T15:57:34,486 copying build/lib/colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-09-06T15:57:34,490 copying build/lib/colossalai/utils/multi_tensor_apply/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-09-06T15:57:34,496 creating build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-09-06T15:57:34,498 copying build/lib/colossalai/utils/rank_recorder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-09-06T15:57:34,502 copying build/lib/colossalai/utils/rank_recorder/rank_recorder.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-09-06T15:57:34,509 creating build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-09-06T15:57:34,511 copying build/lib/colossalai/utils/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-09-06T15:57:34,516 copying build/lib/colossalai/utils/checkpoint/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-09-06T15:57:34,519 copying build/lib/colossalai/utils/checkpoint/module_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-09-06T15:57:34,524 copying build/lib/colossalai/utils/timer.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-09-06T15:57:34,527 running install_egg_info 2023-09-06T15:57:34,662 running egg_info 2023-09-06T15:57:34,718 writing colossalai.egg-info/PKG-INFO 2023-09-06T15:57:34,724 writing dependency_links to colossalai.egg-info/dependency_links.txt 2023-09-06T15:57:34,727 writing entry points to colossalai.egg-info/entry_points.txt 2023-09-06T15:57:34,730 writing requirements to colossalai.egg-info/requires.txt 2023-09-06T15:57:34,733 writing top-level names to colossalai.egg-info/top_level.txt 2023-09-06T15:57:35,108 reading manifest file 'colossalai.egg-info/SOURCES.txt' 2023-09-06T15:57:35,208 reading manifest template 'MANIFEST.in' 2023-09-06T15:57:37,597 warning: no files found matching '*.tr' under directory 'colossalai' 2023-09-06T15:57:38,782 warning: no files found matching '*.cc' under directory 'colossalai' 2023-09-06T15:57:39,368 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-09-06T15:57:39,385 adding license file 'LICENSE' 2023-09-06T15:57:39,501 writing manifest file 'colossalai.egg-info/SOURCES.txt' 2023-09-06T15:57:39,507 Copying colossalai.egg-info to build/bdist.linux-armv7l/wheel/colossalai-0.3.2-py3.7.egg-info 2023-09-06T15:57:39,536 running install_scripts 2023-09-06T15:57:39,593 adding license file "LICENSE" (matched pattern "LICEN[CS]E*") 2023-09-06T15:57:39,603 creating build/bdist.linux-armv7l/wheel/colossalai-0.3.2.dist-info/WHEEL 2023-09-06T15:57:39,608 creating '/tmp/pip-wheel-gra_qedi/colossalai-0.3.2-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2023-09-06T15:57:39,614 adding 'colossalai/__init__.py' 2023-09-06T15:57:39,617 adding 'colossalai/constants.py' 2023-09-06T15:57:39,620 adding 'colossalai/core.py' 2023-09-06T15:57:39,622 adding 'colossalai/global_variables.py' 2023-09-06T15:57:39,628 adding 'colossalai/initialize.py' 2023-09-06T15:57:39,631 adding 'colossalai/version.py' 2023-09-06T15:57:39,634 adding 'colossalai/_C/__init__.py' 2023-09-06T15:57:39,638 adding 'colossalai/_analyzer/__init__.py' 2023-09-06T15:57:39,640 adding 'colossalai/_analyzer/envs.py' 2023-09-06T15:57:39,644 adding 'colossalai/_analyzer/_subclasses/__init__.py' 2023-09-06T15:57:39,649 adding 'colossalai/_analyzer/_subclasses/_meta_registration.py' 2023-09-06T15:57:39,652 adding 'colossalai/_analyzer/_subclasses/_monkey_patch.py' 2023-09-06T15:57:39,658 adding 'colossalai/_analyzer/_subclasses/flop_tensor.py' 2023-09-06T15:57:39,662 adding 'colossalai/_analyzer/_subclasses/meta_tensor.py' 2023-09-06T15:57:39,666 adding 'colossalai/_analyzer/fx/__init__.py' 2023-09-06T15:57:39,672 adding 'colossalai/_analyzer/fx/codegen.py' 2023-09-06T15:57:39,676 adding 'colossalai/_analyzer/fx/graph_module.py' 2023-09-06T15:57:39,680 adding 'colossalai/_analyzer/fx/node_util.py' 2023-09-06T15:57:39,683 adding 'colossalai/_analyzer/fx/symbolic_profile.py' 2023-09-06T15:57:39,686 adding 'colossalai/_analyzer/fx/passes/__init__.py' 2023-09-06T15:57:39,691 adding 'colossalai/_analyzer/fx/passes/graph_profile.py' 2023-09-06T15:57:39,695 adding 'colossalai/_analyzer/fx/passes/shape_prop.py' 2023-09-06T15:57:39,699 adding 'colossalai/_analyzer/fx/tracer/__init__.py' 2023-09-06T15:57:39,702 adding 'colossalai/_analyzer/fx/tracer/bias_addition.py' 2023-09-06T15:57:39,705 adding 'colossalai/_analyzer/fx/tracer/custom_leaf_module.py' 2023-09-06T15:57:39,707 adding 'colossalai/_analyzer/fx/tracer/proxy.py' 2023-09-06T15:57:39,711 adding 'colossalai/_analyzer/fx/tracer/symbolic_trace.py' 2023-09-06T15:57:39,716 adding 'colossalai/_analyzer/fx/tracer/tracer.py' 2023-09-06T15:57:39,720 adding 'colossalai/amp/__init__.py' 2023-09-06T15:57:39,723 adding 'colossalai/amp/amp_type.py' 2023-09-06T15:57:39,726 adding 'colossalai/amp/apex_amp/__init__.py' 2023-09-06T15:57:39,729 adding 'colossalai/amp/apex_amp/apex_amp.py' 2023-09-06T15:57:39,733 adding 'colossalai/amp/naive_amp/__init__.py' 2023-09-06T15:57:39,738 adding 'colossalai/amp/naive_amp/_fp16_optimizer.py' 2023-09-06T15:57:39,741 adding 'colossalai/amp/naive_amp/_utils.py' 2023-09-06T15:57:39,744 adding 'colossalai/amp/naive_amp/mixed_precision_optimizer.py' 2023-09-06T15:57:39,747 adding 'colossalai/amp/naive_amp/naive_amp.py' 2023-09-06T15:57:39,751 adding 'colossalai/amp/naive_amp/grad_scaler/__init__.py' 2023-09-06T15:57:39,754 adding 'colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py' 2023-09-06T15:57:39,756 adding 'colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py' 2023-09-06T15:57:39,759 adding 'colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py' 2023-09-06T15:57:39,763 adding 'colossalai/amp/naive_amp/mixed_precision_mixin/__init__.py' 2023-09-06T15:57:39,765 adding 'colossalai/amp/naive_amp/mixed_precision_mixin/base.py' 2023-09-06T15:57:39,768 adding 'colossalai/amp/naive_amp/mixed_precision_mixin/bf16.py' 2023-09-06T15:57:39,771 adding 'colossalai/amp/naive_amp/mixed_precision_mixin/fp16.py' 2023-09-06T15:57:39,774 adding 'colossalai/amp/torch_amp/__init__.py' 2023-09-06T15:57:39,782 adding 'colossalai/amp/torch_amp/_grad_scaler.py' 2023-09-06T15:57:39,786 adding 'colossalai/amp/torch_amp/torch_amp.py' 2023-09-06T15:57:39,790 adding 'colossalai/auto_parallel/__init__.py' 2023-09-06T15:57:39,793 adding 'colossalai/auto_parallel/checkpoint/__init__.py' 2023-09-06T15:57:39,795 adding 'colossalai/auto_parallel/checkpoint/build_c_ext.py' 2023-09-06T15:57:39,799 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_base.py' 2023-09-06T15:57:39,802 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py' 2023-09-06T15:57:39,808 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py' 2023-09-06T15:57:39,811 adding 'colossalai/auto_parallel/checkpoint/operation.py' 2023-09-06T15:57:39,815 adding 'colossalai/auto_parallel/meta_profiler/__init__.py' 2023-09-06T15:57:39,817 adding 'colossalai/auto_parallel/meta_profiler/constants.py' 2023-09-06T15:57:39,819 adding 'colossalai/auto_parallel/meta_profiler/registry.py' 2023-09-06T15:57:39,822 adding 'colossalai/auto_parallel/meta_profiler/shard_metainfo.py' 2023-09-06T15:57:39,826 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py' 2023-09-06T15:57:39,829 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/activation.py' 2023-09-06T15:57:39,832 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py' 2023-09-06T15:57:39,835 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/conv.py' 2023-09-06T15:57:39,839 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py' 2023-09-06T15:57:39,844 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/linear.py' 2023-09-06T15:57:39,847 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py' 2023-09-06T15:57:39,851 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/norm.py' 2023-09-06T15:57:39,854 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py' 2023-09-06T15:57:39,857 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py' 2023-09-06T15:57:39,860 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/where.py' 2023-09-06T15:57:39,863 adding 'colossalai/auto_parallel/offload/__init__.py' 2023-09-06T15:57:39,867 adding 'colossalai/auto_parallel/offload/amp_optimizer.py' 2023-09-06T15:57:39,870 adding 'colossalai/auto_parallel/offload/base_offload_module.py' 2023-09-06T15:57:39,873 adding 'colossalai/auto_parallel/offload/mem_optimize.py' 2023-09-06T15:57:39,876 adding 'colossalai/auto_parallel/offload/region.py' 2023-09-06T15:57:39,882 adding 'colossalai/auto_parallel/offload/region_manager.py' 2023-09-06T15:57:39,886 adding 'colossalai/auto_parallel/offload/runtime.py' 2023-09-06T15:57:39,892 adding 'colossalai/auto_parallel/offload/solver.py' 2023-09-06T15:57:39,897 adding 'colossalai/auto_parallel/offload/training_simulator.py' 2023-09-06T15:57:39,900 adding 'colossalai/auto_parallel/offload/util.py' 2023-09-06T15:57:39,904 adding 'colossalai/auto_parallel/passes/__init__.py' 2023-09-06T15:57:39,907 adding 'colossalai/auto_parallel/passes/comm_metainfo_pass.py' 2023-09-06T15:57:39,910 adding 'colossalai/auto_parallel/passes/constants.py' 2023-09-06T15:57:39,913 adding 'colossalai/auto_parallel/passes/meta_info_prop.py' 2023-09-06T15:57:39,917 adding 'colossalai/auto_parallel/passes/runtime_apply_pass.py' 2023-09-06T15:57:39,924 adding 'colossalai/auto_parallel/passes/runtime_preparation_pass.py' 2023-09-06T15:57:39,928 adding 'colossalai/auto_parallel/pipeline_shard/__init__.py' 2023-09-06T15:57:39,931 adding 'colossalai/auto_parallel/tensor_shard/__init__.py' 2023-09-06T15:57:39,934 adding 'colossalai/auto_parallel/tensor_shard/constants.py' 2023-09-06T15:57:39,939 adding 'colossalai/auto_parallel/tensor_shard/initialize.py' 2023-09-06T15:57:39,942 adding 'colossalai/auto_parallel/tensor_shard/options.py' 2023-09-06T15:57:39,947 adding 'colossalai/auto_parallel/tensor_shard/sharding_strategy.py' 2023-09-06T15:57:39,952 adding 'colossalai/auto_parallel/tensor_shard/node_handler/__init__.py' 2023-09-06T15:57:39,955 adding 'colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py' 2023-09-06T15:57:39,958 adding 'colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py' 2023-09-06T15:57:39,962 adding 'colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py' 2023-09-06T15:57:39,965 adding 'colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py' 2023-09-06T15:57:39,968 adding 'colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py' 2023-09-06T15:57:39,971 adding 'colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py' 2023-09-06T15:57:39,975 adding 'colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py' 2023-09-06T15:57:39,978 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py' 2023-09-06T15:57:39,981 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py' 2023-09-06T15:57:39,984 adding 'colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py' 2023-09-06T15:57:39,988 adding 'colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py' 2023-09-06T15:57:39,994 adding 'colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py' 2023-09-06T15:57:39,999 adding 'colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py' 2023-09-06T15:57:40,002 adding 'colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py' 2023-09-06T15:57:40,005 adding 'colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py' 2023-09-06T15:57:40,008 adding 'colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py' 2023-09-06T15:57:40,011 adding 'colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py' 2023-09-06T15:57:40,013 adding 'colossalai/auto_parallel/tensor_shard/node_handler/registry.py' 2023-09-06T15:57:40,016 adding 'colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py' 2023-09-06T15:57:40,018 adding 'colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py' 2023-09-06T15:57:40,021 adding 'colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py' 2023-09-06T15:57:40,024 adding 'colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py' 2023-09-06T15:57:40,027 adding 'colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py' 2023-09-06T15:57:40,029 adding 'colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py' 2023-09-06T15:57:40,032 adding 'colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py' 2023-09-06T15:57:40,035 adding 'colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py' 2023-09-06T15:57:40,039 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py' 2023-09-06T15:57:40,043 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py' 2023-09-06T15:57:40,047 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py' 2023-09-06T15:57:40,052 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py' 2023-09-06T15:57:40,056 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py' 2023-09-06T15:57:40,059 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py' 2023-09-06T15:57:40,063 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py' 2023-09-06T15:57:40,067 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py' 2023-09-06T15:57:40,074 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py' 2023-09-06T15:57:40,079 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py' 2023-09-06T15:57:40,082 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py' 2023-09-06T15:57:40,085 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py' 2023-09-06T15:57:40,090 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py' 2023-09-06T15:57:40,093 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py' 2023-09-06T15:57:40,098 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py' 2023-09-06T15:57:40,101 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py' 2023-09-06T15:57:40,104 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py' 2023-09-06T15:57:40,107 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py' 2023-09-06T15:57:40,110 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py' 2023-09-06T15:57:40,114 adding 'colossalai/auto_parallel/tensor_shard/solver/__init__.py' 2023-09-06T15:57:40,118 adding 'colossalai/auto_parallel/tensor_shard/solver/cost_graph.py' 2023-09-06T15:57:40,121 adding 'colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py' 2023-09-06T15:57:40,127 adding 'colossalai/auto_parallel/tensor_shard/solver/solver.py' 2023-09-06T15:57:40,132 adding 'colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py' 2023-09-06T15:57:40,136 adding 'colossalai/auto_parallel/tensor_shard/utils/__init__.py' 2023-09-06T15:57:40,139 adding 'colossalai/auto_parallel/tensor_shard/utils/broadcast.py' 2023-09-06T15:57:40,143 adding 'colossalai/auto_parallel/tensor_shard/utils/factory.py' 2023-09-06T15:57:40,146 adding 'colossalai/auto_parallel/tensor_shard/utils/misc.py' 2023-09-06T15:57:40,150 adding 'colossalai/auto_parallel/tensor_shard/utils/reshape.py' 2023-09-06T15:57:40,153 adding 'colossalai/auto_parallel/tensor_shard/utils/sharding.py' 2023-09-06T15:57:40,157 adding 'colossalai/booster/__init__.py' 2023-09-06T15:57:40,159 adding 'colossalai/booster/accelerator.py' 2023-09-06T15:57:40,163 adding 'colossalai/booster/booster.py' 2023-09-06T15:57:40,168 adding 'colossalai/booster/mixed_precision/__init__.py' 2023-09-06T15:57:40,170 adding 'colossalai/booster/mixed_precision/bf16.py' 2023-09-06T15:57:40,173 adding 'colossalai/booster/mixed_precision/fp16_apex.py' 2023-09-06T15:57:40,175 adding 'colossalai/booster/mixed_precision/fp16_naive.py' 2023-09-06T15:57:40,178 adding 'colossalai/booster/mixed_precision/fp16_torch.py' 2023-09-06T15:57:40,181 adding 'colossalai/booster/mixed_precision/fp8.py' 2023-09-06T15:57:40,184 adding 'colossalai/booster/mixed_precision/mixed_precision_base.py' 2023-09-06T15:57:40,188 adding 'colossalai/booster/plugin/__init__.py' 2023-09-06T15:57:40,190 adding 'colossalai/booster/plugin/dp_plugin_base.py' 2023-09-06T15:57:40,196 adding 'colossalai/booster/plugin/gemini_plugin.py' 2023-09-06T15:57:40,204 adding 'colossalai/booster/plugin/hybrid_parallel_plugin.py' 2023-09-06T15:57:40,209 adding 'colossalai/booster/plugin/low_level_zero_plugin.py' 2023-09-06T15:57:40,212 adding 'colossalai/booster/plugin/plugin_base.py' 2023-09-06T15:57:40,214 adding 'colossalai/booster/plugin/pp_plugin_base.py' 2023-09-06T15:57:40,218 adding 'colossalai/booster/plugin/torch_ddp_plugin.py' 2023-09-06T15:57:40,221 adding 'colossalai/booster/plugin/torch_fsdp_plugin.py' 2023-09-06T15:57:40,225 adding 'colossalai/checkpoint_io/__init__.py' 2023-09-06T15:57:40,229 adding 'colossalai/checkpoint_io/checkpoint_io_base.py' 2023-09-06T15:57:40,233 adding 'colossalai/checkpoint_io/general_checkpoint_io.py' 2023-09-06T15:57:40,242 adding 'colossalai/checkpoint_io/hybrid_parallel_checkpoint_io.py' 2023-09-06T15:57:40,246 adding 'colossalai/checkpoint_io/index_file.py' 2023-09-06T15:57:40,255 adding 'colossalai/checkpoint_io/utils.py' 2023-09-06T15:57:40,259 adding 'colossalai/cli/__init__.py' 2023-09-06T15:57:40,262 adding 'colossalai/cli/cli.py' 2023-09-06T15:57:40,265 adding 'colossalai/cli/benchmark/__init__.py' 2023-09-06T15:57:40,268 adding 'colossalai/cli/benchmark/benchmark.py' 2023-09-06T15:57:40,271 adding 'colossalai/cli/benchmark/models.py' 2023-09-06T15:57:40,274 adding 'colossalai/cli/benchmark/utils.py' 2023-09-06T15:57:40,278 adding 'colossalai/cli/check/__init__.py' 2023-09-06T15:57:40,281 adding 'colossalai/cli/check/check_installation.py' 2023-09-06T15:57:40,286 adding 'colossalai/cli/launcher/__init__.py' 2023-09-06T15:57:40,289 adding 'colossalai/cli/launcher/hostinfo.py' 2023-09-06T15:57:40,292 adding 'colossalai/cli/launcher/multinode_runner.py' 2023-09-06T15:57:40,296 adding 'colossalai/cli/launcher/run.py' 2023-09-06T15:57:40,300 adding 'colossalai/cluster/__init__.py' 2023-09-06T15:57:40,303 adding 'colossalai/cluster/device_mesh_manager.py' 2023-09-06T15:57:40,306 adding 'colossalai/cluster/dist_coordinator.py' 2023-09-06T15:57:40,309 adding 'colossalai/cluster/process_group_manager.py' 2023-09-06T15:57:40,313 adding 'colossalai/cluster/process_group_mesh.py' 2023-09-06T15:57:40,317 adding 'colossalai/communication/__init__.py' 2023-09-06T15:57:40,321 adding 'colossalai/communication/collective.py' 2023-09-06T15:57:40,325 adding 'colossalai/communication/p2p.py' 2023-09-06T15:57:40,330 adding 'colossalai/communication/p2p_v2.py' 2023-09-06T15:57:40,332 adding 'colossalai/communication/ring.py' 2023-09-06T15:57:40,336 adding 'colossalai/communication/utils.py' 2023-09-06T15:57:40,339 adding 'colossalai/context/__init__.py' 2023-09-06T15:57:40,342 adding 'colossalai/context/config.py' 2023-09-06T15:57:40,345 adding 'colossalai/context/moe_context.py' 2023-09-06T15:57:40,352 adding 'colossalai/context/parallel_context.py' 2023-09-06T15:57:40,355 adding 'colossalai/context/parallel_mode.py' 2023-09-06T15:57:40,358 adding 'colossalai/context/singleton_meta.py' 2023-09-06T15:57:40,362 adding 'colossalai/context/process_group_initializer/__init__.py' 2023-09-06T15:57:40,364 adding 'colossalai/context/process_group_initializer/initializer_1d.py' 2023-09-06T15:57:40,367 adding 'colossalai/context/process_group_initializer/initializer_2d.py' 2023-09-06T15:57:40,371 adding 'colossalai/context/process_group_initializer/initializer_2p5d.py' 2023-09-06T15:57:40,375 adding 'colossalai/context/process_group_initializer/initializer_3d.py' 2023-09-06T15:57:40,378 adding 'colossalai/context/process_group_initializer/initializer_data.py' 2023-09-06T15:57:40,381 adding 'colossalai/context/process_group_initializer/initializer_model.py' 2023-09-06T15:57:40,384 adding 'colossalai/context/process_group_initializer/initializer_pipeline.py' 2023-09-06T15:57:40,387 adding 'colossalai/context/process_group_initializer/initializer_sequence.py' 2023-09-06T15:57:40,390 adding 'colossalai/context/process_group_initializer/initializer_tensor.py' 2023-09-06T15:57:40,392 adding 'colossalai/context/process_group_initializer/process_group_initializer.py' 2023-09-06T15:57:40,396 adding 'colossalai/context/random/__init__.py' 2023-09-06T15:57:40,399 adding 'colossalai/context/random/_helper.py' 2023-09-06T15:57:40,402 adding 'colossalai/context/random/seed_manager.py' 2023-09-06T15:57:40,405 adding 'colossalai/device/__init__.py' 2023-09-06T15:57:40,411 adding 'colossalai/device/alpha_beta_profiler.py' 2023-09-06T15:57:40,414 adding 'colossalai/device/calc_pipeline_strategy.py' 2023-09-06T15:57:40,420 adding 'colossalai/device/device_mesh.py' 2023-09-06T15:57:40,425 adding 'colossalai/fx/__init__.py' 2023-09-06T15:57:40,427 adding 'colossalai/fx/_compatibility.py' 2023-09-06T15:57:40,433 adding 'colossalai/fx/_meta_regist_12.py' 2023-09-06T15:57:40,436 adding 'colossalai/fx/_meta_regist_13.py' 2023-09-06T15:57:40,440 adding 'colossalai/fx/graph_module.py' 2023-09-06T15:57:40,442 adding 'colossalai/fx/proxy.py' 2023-09-06T15:57:40,446 adding 'colossalai/fx/codegen/__init__.py' 2023-09-06T15:57:40,456 adding 'colossalai/fx/codegen/activation_checkpoint_codegen.py' 2023-09-06T15:57:40,461 adding 'colossalai/fx/passes/__init__.py' 2023-09-06T15:57:40,466 adding 'colossalai/fx/passes/adding_split_node_pass.py' 2023-09-06T15:57:40,470 adding 'colossalai/fx/passes/concrete_info_prop.py' 2023-09-06T15:57:40,475 adding 'colossalai/fx/passes/meta_info_prop.py' 2023-09-06T15:57:40,481 adding 'colossalai/fx/passes/passes_for_gpt2_test.py' 2023-09-06T15:57:40,485 adding 'colossalai/fx/passes/shard_1d_pass.py' 2023-09-06T15:57:40,490 adding 'colossalai/fx/passes/split_module.py' 2023-09-06T15:57:40,494 adding 'colossalai/fx/passes/utils.py' 2023-09-06T15:57:40,498 adding 'colossalai/fx/profiler/__init__.py' 2023-09-06T15:57:40,500 adding 'colossalai/fx/profiler/constants.py' 2023-09-06T15:57:40,504 adding 'colossalai/fx/profiler/dataflow.py' 2023-09-06T15:57:40,507 adding 'colossalai/fx/profiler/memory_utils.py' 2023-09-06T15:57:40,511 adding 'colossalai/fx/profiler/opcount.py' 2023-09-06T15:57:40,516 adding 'colossalai/fx/profiler/profiler.py' 2023-09-06T15:57:40,519 adding 'colossalai/fx/profiler/shard_utils.py' 2023-09-06T15:57:40,522 adding 'colossalai/fx/profiler/tensor.py' 2023-09-06T15:57:40,526 adding 'colossalai/fx/profiler/experimental/__init__.py' 2023-09-06T15:57:40,529 adding 'colossalai/fx/profiler/experimental/constants.py' 2023-09-06T15:57:40,532 adding 'colossalai/fx/profiler/experimental/profiler.py' 2023-09-06T15:57:40,535 adding 'colossalai/fx/profiler/experimental/registry.py' 2023-09-06T15:57:40,537 adding 'colossalai/fx/profiler/experimental/shard_utils.py' 2023-09-06T15:57:40,541 adding 'colossalai/fx/profiler/experimental/profiler_function/__init__.py' 2023-09-06T15:57:40,544 adding 'colossalai/fx/profiler/experimental/profiler_function/activation_function.py' 2023-09-06T15:57:40,547 adding 'colossalai/fx/profiler/experimental/profiler_function/arithmetic.py' 2023-09-06T15:57:40,549 adding 'colossalai/fx/profiler/experimental/profiler_function/embedding.py' 2023-09-06T15:57:40,552 adding 'colossalai/fx/profiler/experimental/profiler_function/linear.py' 2023-09-06T15:57:40,554 adding 'colossalai/fx/profiler/experimental/profiler_function/normalization.py' 2023-09-06T15:57:40,557 adding 'colossalai/fx/profiler/experimental/profiler_function/pooling.py' 2023-09-06T15:57:40,559 adding 'colossalai/fx/profiler/experimental/profiler_function/python_ops.py' 2023-09-06T15:57:40,562 adding 'colossalai/fx/profiler/experimental/profiler_function/torch_ops.py' 2023-09-06T15:57:40,566 adding 'colossalai/fx/profiler/experimental/profiler_module/__init__.py' 2023-09-06T15:57:40,569 adding 'colossalai/fx/profiler/experimental/profiler_module/activation_function.py' 2023-09-06T15:57:40,571 adding 'colossalai/fx/profiler/experimental/profiler_module/attention.py' 2023-09-06T15:57:40,574 adding 'colossalai/fx/profiler/experimental/profiler_module/convolution.py' 2023-09-06T15:57:40,577 adding 'colossalai/fx/profiler/experimental/profiler_module/dropout.py' 2023-09-06T15:57:40,580 adding 'colossalai/fx/profiler/experimental/profiler_module/embedding.py' 2023-09-06T15:57:40,582 adding 'colossalai/fx/profiler/experimental/profiler_module/linear.py' 2023-09-06T15:57:40,585 adding 'colossalai/fx/profiler/experimental/profiler_module/normalization.py' 2023-09-06T15:57:40,587 adding 'colossalai/fx/profiler/experimental/profiler_module/pooling.py' 2023-09-06T15:57:40,590 adding 'colossalai/fx/profiler/experimental/profiler_module/rnn.py' 2023-09-06T15:57:40,593 adding 'colossalai/fx/profiler/experimental/profiler_module/torch_op.py' 2023-09-06T15:57:40,596 adding 'colossalai/fx/tracer/__init__.py' 2023-09-06T15:57:40,600 adding 'colossalai/fx/tracer/_meta_trace.py' 2023-09-06T15:57:40,602 adding 'colossalai/fx/tracer/_symbolic_trace.py' 2023-09-06T15:57:40,605 adding 'colossalai/fx/tracer/_tracer_utils.py' 2023-09-06T15:57:40,612 adding 'colossalai/fx/tracer/experimental.py' 2023-09-06T15:57:40,615 adding 'colossalai/fx/tracer/registry.py' 2023-09-06T15:57:40,623 adding 'colossalai/fx/tracer/tracer.py' 2023-09-06T15:57:40,627 adding 'colossalai/fx/tracer/bias_addition_patch/__init__.py' 2023-09-06T15:57:40,630 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py' 2023-09-06T15:57:40,633 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py' 2023-09-06T15:57:40,636 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py' 2023-09-06T15:57:40,639 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py' 2023-09-06T15:57:40,641 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py' 2023-09-06T15:57:40,645 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py' 2023-09-06T15:57:40,648 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py' 2023-09-06T15:57:40,651 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py' 2023-09-06T15:57:40,653 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py' 2023-09-06T15:57:40,657 adding 'colossalai/fx/tracer/meta_patch/__init__.py' 2023-09-06T15:57:40,660 adding 'colossalai/fx/tracer/meta_patch/patched_function/__init__.py' 2023-09-06T15:57:40,663 adding 'colossalai/fx/tracer/meta_patch/patched_function/activation_function.py' 2023-09-06T15:57:40,665 adding 'colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py' 2023-09-06T15:57:40,668 adding 'colossalai/fx/tracer/meta_patch/patched_function/convolution.py' 2023-09-06T15:57:40,671 adding 'colossalai/fx/tracer/meta_patch/patched_function/embedding.py' 2023-09-06T15:57:40,673 adding 'colossalai/fx/tracer/meta_patch/patched_function/normalization.py' 2023-09-06T15:57:40,676 adding 'colossalai/fx/tracer/meta_patch/patched_function/python_ops.py' 2023-09-06T15:57:40,679 adding 'colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py' 2023-09-06T15:57:40,683 adding 'colossalai/fx/tracer/meta_patch/patched_module/__init__.py' 2023-09-06T15:57:40,686 adding 'colossalai/fx/tracer/meta_patch/patched_module/activation_function.py' 2023-09-06T15:57:40,689 adding 'colossalai/fx/tracer/meta_patch/patched_module/convolution.py' 2023-09-06T15:57:40,691 adding 'colossalai/fx/tracer/meta_patch/patched_module/embedding.py' 2023-09-06T15:57:40,693 adding 'colossalai/fx/tracer/meta_patch/patched_module/linear.py' 2023-09-06T15:57:40,696 adding 'colossalai/fx/tracer/meta_patch/patched_module/normalization.py' 2023-09-06T15:57:40,699 adding 'colossalai/fx/tracer/meta_patch/patched_module/pooling.py' 2023-09-06T15:57:40,701 adding 'colossalai/fx/tracer/meta_patch/patched_module/rnn.py' 2023-09-06T15:57:40,705 adding 'colossalai/interface/__init__.py' 2023-09-06T15:57:40,707 adding 'colossalai/interface/model.py' 2023-09-06T15:57:40,710 adding 'colossalai/interface/optimizer.py' 2023-09-06T15:57:40,714 adding 'colossalai/kernel/__init__.py' 2023-09-06T15:57:40,717 adding 'colossalai/kernel/cuda_native/__init__.py' 2023-09-06T15:57:40,720 adding 'colossalai/kernel/cuda_native/layer_norm.py' 2023-09-06T15:57:40,724 adding 'colossalai/kernel/cuda_native/multihead_attention.py' 2023-09-06T15:57:40,727 adding 'colossalai/kernel/cuda_native/scaled_softmax.py' 2023-09-06T15:57:40,732 adding 'colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp' 2023-09-06T15:57:40,735 adding 'colossalai/kernel/cuda_native/csrc/compat.h' 2023-09-06T15:57:40,739 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.cpp' 2023-09-06T15:57:40,743 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.h' 2023-09-06T15:57:40,746 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp' 2023-09-06T15:57:40,753 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu' 2023-09-06T15:57:40,756 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda.cpp' 2023-09-06T15:57:40,762 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu' 2023-09-06T15:57:40,765 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu' 2023-09-06T15:57:40,769 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh' 2023-09-06T15:57:40,773 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu' 2023-09-06T15:57:40,777 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu' 2023-09-06T15:57:40,781 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu' 2023-09-06T15:57:40,784 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu' 2023-09-06T15:57:40,789 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp' 2023-09-06T15:57:40,792 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h' 2023-09-06T15:57:40,795 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp' 2023-09-06T15:57:40,800 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h' 2023-09-06T15:57:40,803 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu' 2023-09-06T15:57:40,806 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp' 2023-09-06T15:57:40,810 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h' 2023-09-06T15:57:40,813 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu' 2023-09-06T15:57:40,817 adding 'colossalai/kernel/cuda_native/csrc/type_shim.h' 2023-09-06T15:57:40,823 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h' 2023-09-06T15:57:40,826 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/context.h' 2023-09-06T15:57:40,828 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h' 2023-09-06T15:57:40,831 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h' 2023-09-06T15:57:40,833 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h' 2023-09-06T15:57:40,836 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h' 2023-09-06T15:57:40,839 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h' 2023-09-06T15:57:40,843 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h' 2023-09-06T15:57:40,845 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh' 2023-09-06T15:57:40,848 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h' 2023-09-06T15:57:40,850 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h' 2023-09-06T15:57:40,853 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h' 2023-09-06T15:57:40,857 adding 'colossalai/kernel/cuda_native/mha/__init__.py' 2023-09-06T15:57:40,859 adding 'colossalai/kernel/cuda_native/mha/flash_attn_2.py' 2023-09-06T15:57:40,862 adding 'colossalai/kernel/cuda_native/mha/mem_eff_attn.py' 2023-09-06T15:57:40,865 adding 'colossalai/kernel/cuda_native/mha/mha.py' 2023-09-06T15:57:40,868 adding 'colossalai/kernel/cuda_native/mha/utils.py' 2023-09-06T15:57:40,871 adding 'colossalai/kernel/jit/__init__.py' 2023-09-06T15:57:40,874 adding 'colossalai/kernel/jit/bias_dropout_add.py' 2023-09-06T15:57:40,876 adding 'colossalai/kernel/jit/bias_gelu.py' 2023-09-06T15:57:40,879 adding 'colossalai/kernel/jit/option.py' 2023-09-06T15:57:40,883 adding 'colossalai/kernel/op_builder/__init__.py' 2023-09-06T15:57:40,887 adding 'colossalai/kernel/op_builder/builder.py' 2023-09-06T15:57:40,890 adding 'colossalai/kernel/op_builder/cpu_adam.py' 2023-09-06T15:57:40,892 adding 'colossalai/kernel/op_builder/fused_optim.py' 2023-09-06T15:57:40,895 adding 'colossalai/kernel/op_builder/layernorm.py' 2023-09-06T15:57:40,897 adding 'colossalai/kernel/op_builder/moe.py' 2023-09-06T15:57:40,900 adding 'colossalai/kernel/op_builder/multi_head_attn.py' 2023-09-06T15:57:40,902 adding 'colossalai/kernel/op_builder/scaled_masked_softmax.py' 2023-09-06T15:57:40,905 adding 'colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py' 2023-09-06T15:57:40,908 adding 'colossalai/kernel/op_builder/utils.py' 2023-09-06T15:57:40,912 adding 'colossalai/lazy/__init__.py' 2023-09-06T15:57:40,919 adding 'colossalai/lazy/lazy_init.py' 2023-09-06T15:57:40,923 adding 'colossalai/legacy/__init__.py' 2023-09-06T15:57:40,926 adding 'colossalai/legacy/builder/__init__.py' 2023-09-06T15:57:40,929 adding 'colossalai/legacy/builder/builder.py' 2023-09-06T15:57:40,932 adding 'colossalai/legacy/engine/__init__.py' 2023-09-06T15:57:40,936 adding 'colossalai/legacy/engine/_base_engine.py' 2023-09-06T15:57:40,940 adding 'colossalai/legacy/engine/gradient_accumulation/__init__.py' 2023-09-06T15:57:40,944 adding 'colossalai/legacy/engine/gradient_accumulation/_gradient_accumulation.py' 2023-09-06T15:57:40,948 adding 'colossalai/legacy/engine/gradient_handler/__init__.py' 2023-09-06T15:57:40,950 adding 'colossalai/legacy/engine/gradient_handler/_base_gradient_handler.py' 2023-09-06T15:57:40,953 adding 'colossalai/legacy/engine/gradient_handler/_data_parallel_gradient_handler.py' 2023-09-06T15:57:40,956 adding 'colossalai/legacy/engine/gradient_handler/_moe_gradient_handler.py' 2023-09-06T15:57:40,958 adding 'colossalai/legacy/engine/gradient_handler/_pipeline_parallel_gradient_handler.py' 2023-09-06T15:57:40,961 adding 'colossalai/legacy/engine/gradient_handler/_sequence_parallel_gradient_handler.py' 2023-09-06T15:57:40,963 adding 'colossalai/legacy/engine/gradient_handler/_zero_gradient_handler.py' 2023-09-06T15:57:40,966 adding 'colossalai/legacy/engine/gradient_handler/utils.py' 2023-09-06T15:57:40,970 adding 'colossalai/legacy/engine/schedule/__init__.py' 2023-09-06T15:57:40,973 adding 'colossalai/legacy/engine/schedule/_base_schedule.py' 2023-09-06T15:57:40,976 adding 'colossalai/legacy/engine/schedule/_non_pipeline_schedule.py' 2023-09-06T15:57:40,985 adding 'colossalai/legacy/engine/schedule/_pipeline_schedule.py' 2023-09-06T15:57:40,990 adding 'colossalai/legacy/engine/schedule/_pipeline_schedule_v2.py' 2023-09-06T15:57:40,993 adding 'colossalai/legacy/registry/__init__.py' 2023-09-06T15:57:40,996 adding 'colossalai/legacy/registry/registry.py' 2023-09-06T15:57:40,999 adding 'colossalai/legacy/trainer/__init__.py' 2023-09-06T15:57:41,004 adding 'colossalai/legacy/trainer/_trainer.py' 2023-09-06T15:57:41,008 adding 'colossalai/legacy/trainer/hooks/__init__.py' 2023-09-06T15:57:41,010 adding 'colossalai/legacy/trainer/hooks/_base_hook.py' 2023-09-06T15:57:41,013 adding 'colossalai/legacy/trainer/hooks/_checkpoint_hook.py' 2023-09-06T15:57:41,016 adding 'colossalai/legacy/trainer/hooks/_commons_.py' 2023-09-06T15:57:41,020 adding 'colossalai/legacy/trainer/hooks/_log_hook.py' 2023-09-06T15:57:41,023 adding 'colossalai/legacy/trainer/hooks/_lr_scheduler_hook.py' 2023-09-06T15:57:41,027 adding 'colossalai/legacy/trainer/hooks/_metric_hook.py' 2023-09-06T15:57:41,032 adding 'colossalai/logging/__init__.py' 2023-09-06T15:57:41,035 adding 'colossalai/logging/logger.py' 2023-09-06T15:57:41,039 adding 'colossalai/nn/__init__.py' 2023-09-06T15:57:41,042 adding 'colossalai/nn/init.py' 2023-09-06T15:57:41,046 adding 'colossalai/nn/_ops/__init__.py' 2023-09-06T15:57:41,050 adding 'colossalai/nn/_ops/_utils.py' 2023-09-06T15:57:41,053 adding 'colossalai/nn/_ops/addmm.py' 2023-09-06T15:57:41,056 adding 'colossalai/nn/_ops/batch_norm.py' 2023-09-06T15:57:41,059 adding 'colossalai/nn/_ops/element_wise.py' 2023-09-06T15:57:41,062 adding 'colossalai/nn/_ops/embedding.py' 2023-09-06T15:57:41,066 adding 'colossalai/nn/_ops/embedding_bag.py' 2023-09-06T15:57:41,069 adding 'colossalai/nn/_ops/layernorm.py' 2023-09-06T15:57:41,073 adding 'colossalai/nn/_ops/linear.py' 2023-09-06T15:57:41,076 adding 'colossalai/nn/_ops/loss.py' 2023-09-06T15:57:41,079 adding 'colossalai/nn/_ops/view.py' 2023-09-06T15:57:41,082 adding 'colossalai/nn/layer/__init__.py' 2023-09-06T15:57:41,085 adding 'colossalai/nn/layer/base_layer.py' 2023-09-06T15:57:41,089 adding 'colossalai/nn/layer/colossalai_layer/__init__.py' 2023-09-06T15:57:41,091 adding 'colossalai/nn/layer/colossalai_layer/_utils.py' 2023-09-06T15:57:41,094 adding 'colossalai/nn/layer/colossalai_layer/dropout.py' 2023-09-06T15:57:41,097 adding 'colossalai/nn/layer/colossalai_layer/embedding.py' 2023-09-06T15:57:41,101 adding 'colossalai/nn/layer/colossalai_layer/linear.py' 2023-09-06T15:57:41,103 adding 'colossalai/nn/layer/colossalai_layer/normalization.py' 2023-09-06T15:57:41,107 adding 'colossalai/nn/layer/moe/__init__.py' 2023-09-06T15:57:41,110 adding 'colossalai/nn/layer/moe/_operation.py' 2023-09-06T15:57:41,113 adding 'colossalai/nn/layer/moe/checkpoint.py' 2023-09-06T15:57:41,117 adding 'colossalai/nn/layer/moe/experts.py' 2023-09-06T15:57:41,120 adding 'colossalai/nn/layer/moe/layers.py' 2023-09-06T15:57:41,124 adding 'colossalai/nn/layer/moe/routers.py' 2023-09-06T15:57:41,127 adding 'colossalai/nn/layer/moe/utils.py' 2023-09-06T15:57:41,131 adding 'colossalai/nn/layer/parallel_1d/__init__.py' 2023-09-06T15:57:41,134 adding 'colossalai/nn/layer/parallel_1d/_operation.py' 2023-09-06T15:57:41,137 adding 'colossalai/nn/layer/parallel_1d/_utils.py' 2023-09-06T15:57:41,146 adding 'colossalai/nn/layer/parallel_1d/layers.py' 2023-09-06T15:57:41,150 adding 'colossalai/nn/layer/parallel_2d/__init__.py' 2023-09-06T15:57:41,157 adding 'colossalai/nn/layer/parallel_2d/_operation.py' 2023-09-06T15:57:41,161 adding 'colossalai/nn/layer/parallel_2d/_utils.py' 2023-09-06T15:57:41,169 adding 'colossalai/nn/layer/parallel_2d/layers.py' 2023-09-06T15:57:41,173 adding 'colossalai/nn/layer/parallel_2p5d/__init__.py' 2023-09-06T15:57:41,181 adding 'colossalai/nn/layer/parallel_2p5d/_operation.py' 2023-09-06T15:57:41,184 adding 'colossalai/nn/layer/parallel_2p5d/_utils.py' 2023-09-06T15:57:41,193 adding 'colossalai/nn/layer/parallel_2p5d/layers.py' 2023-09-06T15:57:41,198 adding 'colossalai/nn/layer/parallel_3d/__init__.py' 2023-09-06T15:57:41,203 adding 'colossalai/nn/layer/parallel_3d/_operation.py' 2023-09-06T15:57:41,206 adding 'colossalai/nn/layer/parallel_3d/_utils.py' 2023-09-06T15:57:41,214 adding 'colossalai/nn/layer/parallel_3d/layers.py' 2023-09-06T15:57:41,219 adding 'colossalai/nn/layer/parallel_sequence/__init__.py' 2023-09-06T15:57:41,222 adding 'colossalai/nn/layer/parallel_sequence/_operation.py' 2023-09-06T15:57:41,224 adding 'colossalai/nn/layer/parallel_sequence/_utils.py' 2023-09-06T15:57:41,228 adding 'colossalai/nn/layer/parallel_sequence/layers.py' 2023-09-06T15:57:41,232 adding 'colossalai/nn/layer/utils/__init__.py' 2023-09-06T15:57:41,235 adding 'colossalai/nn/layer/utils/common.py' 2023-09-06T15:57:41,239 adding 'colossalai/nn/layer/vanilla/__init__.py' 2023-09-06T15:57:41,243 adding 'colossalai/nn/layer/vanilla/layers.py' 2023-09-06T15:57:41,247 adding 'colossalai/nn/layer/wrapper/__init__.py' 2023-09-06T15:57:41,249 adding 'colossalai/nn/layer/wrapper/pipeline_wrapper.py' 2023-09-06T15:57:41,253 adding 'colossalai/nn/loss/__init__.py' 2023-09-06T15:57:41,256 adding 'colossalai/nn/loss/loss_1d.py' 2023-09-06T15:57:41,260 adding 'colossalai/nn/loss/loss_2d.py' 2023-09-06T15:57:41,263 adding 'colossalai/nn/loss/loss_2p5d.py' 2023-09-06T15:57:41,266 adding 'colossalai/nn/loss/loss_3d.py' 2023-09-06T15:57:41,269 adding 'colossalai/nn/loss/loss_moe.py' 2023-09-06T15:57:41,273 adding 'colossalai/nn/lr_scheduler/__init__.py' 2023-09-06T15:57:41,276 adding 'colossalai/nn/lr_scheduler/cosine.py' 2023-09-06T15:57:41,279 adding 'colossalai/nn/lr_scheduler/delayed.py' 2023-09-06T15:57:41,282 adding 'colossalai/nn/lr_scheduler/linear.py' 2023-09-06T15:57:41,285 adding 'colossalai/nn/lr_scheduler/multistep.py' 2023-09-06T15:57:41,288 adding 'colossalai/nn/lr_scheduler/onecycle.py' 2023-09-06T15:57:41,291 adding 'colossalai/nn/lr_scheduler/poly.py' 2023-09-06T15:57:41,294 adding 'colossalai/nn/lr_scheduler/torch.py' 2023-09-06T15:57:41,298 adding 'colossalai/nn/metric/__init__.py' 2023-09-06T15:57:41,300 adding 'colossalai/nn/metric/_utils.py' 2023-09-06T15:57:41,303 adding 'colossalai/nn/metric/accuracy_2d.py' 2023-09-06T15:57:41,305 adding 'colossalai/nn/metric/accuracy_2p5d.py' 2023-09-06T15:57:41,308 adding 'colossalai/nn/metric/accuracy_3d.py' 2023-09-06T15:57:41,311 adding 'colossalai/nn/optimizer/__init__.py' 2023-09-06T15:57:41,314 adding 'colossalai/nn/optimizer/colossalai_optimizer.py' 2023-09-06T15:57:41,317 adding 'colossalai/nn/optimizer/cpu_adam.py' 2023-09-06T15:57:41,321 adding 'colossalai/nn/optimizer/fused_adam.py' 2023-09-06T15:57:41,325 adding 'colossalai/nn/optimizer/fused_lamb.py' 2023-09-06T15:57:41,328 adding 'colossalai/nn/optimizer/fused_sgd.py' 2023-09-06T15:57:41,332 adding 'colossalai/nn/optimizer/hybrid_adam.py' 2023-09-06T15:57:41,335 adding 'colossalai/nn/optimizer/lamb.py' 2023-09-06T15:57:41,338 adding 'colossalai/nn/optimizer/lars.py' 2023-09-06T15:57:41,342 adding 'colossalai/nn/optimizer/nvme_optimizer.py' 2023-09-06T15:57:41,345 adding 'colossalai/nn/parallel/__init__.py' 2023-09-06T15:57:41,349 adding 'colossalai/nn/parallel/data_parallel.py' 2023-09-06T15:57:41,352 adding 'colossalai/nn/parallel/reducer.py' 2023-09-06T15:57:41,356 adding 'colossalai/nn/parallel/layers/__init__.py' 2023-09-06T15:57:41,358 adding 'colossalai/nn/parallel/layers/colo_module.py' 2023-09-06T15:57:41,361 adding 'colossalai/nn/parallel/layers/embedding.py' 2023-09-06T15:57:41,363 adding 'colossalai/nn/parallel/layers/linear.py' 2023-09-06T15:57:41,367 adding 'colossalai/nn/parallel/layers/module_utils.py' 2023-09-06T15:57:41,370 adding 'colossalai/nn/parallel/layers/cache_embedding/__init__.py' 2023-09-06T15:57:41,373 adding 'colossalai/nn/parallel/layers/cache_embedding/base_embedding.py' 2023-09-06T15:57:41,381 adding 'colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py' 2023-09-06T15:57:41,385 adding 'colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py' 2023-09-06T15:57:41,388 adding 'colossalai/nn/parallel/layers/cache_embedding/copyer.py' 2023-09-06T15:57:41,391 adding 'colossalai/nn/parallel/layers/cache_embedding/embedding_config.py' 2023-09-06T15:57:41,394 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py' 2023-09-06T15:57:41,398 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py' 2023-09-06T15:57:41,401 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py' 2023-09-06T15:57:41,405 adding 'colossalai/pipeline/__init__.py' 2023-09-06T15:57:41,408 adding 'colossalai/pipeline/layer_spec.py' 2023-09-06T15:57:41,411 adding 'colossalai/pipeline/p2p.py' 2023-09-06T15:57:41,416 adding 'colossalai/pipeline/pipelinable.py' 2023-09-06T15:57:41,419 adding 'colossalai/pipeline/pipeline_process_group.py' 2023-09-06T15:57:41,422 adding 'colossalai/pipeline/stage_manager.py' 2023-09-06T15:57:41,426 adding 'colossalai/pipeline/utils.py' 2023-09-06T15:57:41,430 adding 'colossalai/pipeline/middleware/__init__.py' 2023-09-06T15:57:41,434 adding 'colossalai/pipeline/middleware/topo.py' 2023-09-06T15:57:41,437 adding 'colossalai/pipeline/middleware/adaptor/__init__.py' 2023-09-06T15:57:41,440 adding 'colossalai/pipeline/middleware/adaptor/fx.py' 2023-09-06T15:57:41,444 adding 'colossalai/pipeline/rpc/__init__.py' 2023-09-06T15:57:41,459 adding 'colossalai/pipeline/rpc/_pipeline_base.py' 2023-09-06T15:57:41,465 adding 'colossalai/pipeline/rpc/_pipeline_schedule.py' 2023-09-06T15:57:41,469 adding 'colossalai/pipeline/rpc/utils.py' 2023-09-06T15:57:41,472 adding 'colossalai/pipeline/schedule/__init__.py' 2023-09-06T15:57:41,476 adding 'colossalai/pipeline/schedule/_utils.py' 2023-09-06T15:57:41,478 adding 'colossalai/pipeline/schedule/base.py' 2023-09-06T15:57:41,483 adding 'colossalai/pipeline/schedule/interleaved_pp.py' 2023-09-06T15:57:41,488 adding 'colossalai/pipeline/schedule/one_f_one_b.py' 2023-09-06T15:57:41,492 adding 'colossalai/shardformer/__init__.py' 2023-09-06T15:57:41,495 adding 'colossalai/shardformer/_utils.py' 2023-09-06T15:57:41,498 adding 'colossalai/shardformer/layer/__init__.py' 2023-09-06T15:57:41,503 adding 'colossalai/shardformer/layer/_operation.py' 2023-09-06T15:57:41,507 adding 'colossalai/shardformer/layer/dropout.py' 2023-09-06T15:57:41,511 adding 'colossalai/shardformer/layer/embedding.py' 2023-09-06T15:57:41,516 adding 'colossalai/shardformer/layer/linear.py' 2023-09-06T15:57:41,519 adding 'colossalai/shardformer/layer/loss.py' 2023-09-06T15:57:41,522 adding 'colossalai/shardformer/layer/normalization.py' 2023-09-06T15:57:41,526 adding 'colossalai/shardformer/layer/parallel_module.py' 2023-09-06T15:57:41,533 adding 'colossalai/shardformer/layer/qkv_fused_linear.py' 2023-09-06T15:57:41,537 adding 'colossalai/shardformer/layer/utils.py' 2023-09-06T15:57:41,541 adding 'colossalai/shardformer/modeling/__init__.py' 2023-09-06T15:57:41,554 adding 'colossalai/shardformer/modeling/bert.py' 2023-09-06T15:57:41,558 adding 'colossalai/shardformer/modeling/blip2.py' 2023-09-06T15:57:41,569 adding 'colossalai/shardformer/modeling/bloom.py' 2023-09-06T15:57:41,575 adding 'colossalai/shardformer/modeling/chatglm2.py' 2023-09-06T15:57:41,586 adding 'colossalai/shardformer/modeling/gpt2.py' 2023-09-06T15:57:41,589 adding 'colossalai/shardformer/modeling/jit.py' 2023-09-06T15:57:41,595 adding 'colossalai/shardformer/modeling/llama.py' 2023-09-06T15:57:41,603 adding 'colossalai/shardformer/modeling/opt.py' 2023-09-06T15:57:41,607 adding 'colossalai/shardformer/modeling/sam.py' 2023-09-06T15:57:41,615 adding 'colossalai/shardformer/modeling/t5.py' 2023-09-06T15:57:41,621 adding 'colossalai/shardformer/modeling/vit.py' 2023-09-06T15:57:41,632 adding 'colossalai/shardformer/modeling/whisper.py' 2023-09-06T15:57:41,637 adding 'colossalai/shardformer/policies/__init__.py' 2023-09-06T15:57:41,640 adding 'colossalai/shardformer/policies/auto_policy.py' 2023-09-06T15:57:41,644 adding 'colossalai/shardformer/policies/base_policy.py' 2023-09-06T15:57:41,649 adding 'colossalai/shardformer/policies/bert.py' 2023-09-06T15:57:41,654 adding 'colossalai/shardformer/policies/blip2.py' 2023-09-06T15:57:41,658 adding 'colossalai/shardformer/policies/bloom.py' 2023-09-06T15:57:41,662 adding 'colossalai/shardformer/policies/chatglm2.py' 2023-09-06T15:57:41,667 adding 'colossalai/shardformer/policies/gpt2.py' 2023-09-06T15:57:41,671 adding 'colossalai/shardformer/policies/llama.py' 2023-09-06T15:57:41,675 adding 'colossalai/shardformer/policies/opt.py' 2023-09-06T15:57:41,678 adding 'colossalai/shardformer/policies/sam.py' 2023-09-06T15:57:41,684 adding 'colossalai/shardformer/policies/t5.py' 2023-09-06T15:57:41,688 adding 'colossalai/shardformer/policies/vit.py' 2023-09-06T15:57:41,693 adding 'colossalai/shardformer/policies/whisper.py' 2023-09-06T15:57:41,697 adding 'colossalai/shardformer/shard/__init__.py' 2023-09-06T15:57:41,700 adding 'colossalai/shardformer/shard/shard_config.py' 2023-09-06T15:57:41,704 adding 'colossalai/shardformer/shard/sharder.py' 2023-09-06T15:57:41,706 adding 'colossalai/shardformer/shard/shardformer.py' 2023-09-06T15:57:41,709 adding 'colossalai/shardformer/shard/utils.py' 2023-09-06T15:57:41,713 adding 'colossalai/tensor/__init__.py' 2023-09-06T15:57:41,716 adding 'colossalai/tensor/colo_parameter.py' 2023-09-06T15:57:41,719 adding 'colossalai/tensor/colo_tensor.py' 2023-09-06T15:57:41,724 adding 'colossalai/tensor/comm_spec.py' 2023-09-06T15:57:41,727 adding 'colossalai/tensor/compute_spec.py' 2023-09-06T15:57:41,729 adding 'colossalai/tensor/const.py' 2023-09-06T15:57:41,733 adding 'colossalai/tensor/dist_spec_mgr.py' 2023-09-06T15:57:41,736 adding 'colossalai/tensor/distspec.py' 2023-09-06T15:57:41,739 adding 'colossalai/tensor/op_wrapper.py' 2023-09-06T15:57:41,742 adding 'colossalai/tensor/param_op_hook.py' 2023-09-06T15:57:41,746 adding 'colossalai/tensor/process_group.py' 2023-09-06T15:57:41,754 adding 'colossalai/tensor/shape_consistency.py' 2023-09-06T15:57:41,759 adding 'colossalai/tensor/sharding_spec.py' 2023-09-06T15:57:41,762 adding 'colossalai/tensor/tensor_spec.py' 2023-09-06T15:57:41,766 adding 'colossalai/tensor/utils.py' 2023-09-06T15:57:41,770 adding 'colossalai/tensor/d_tensor/__init__.py' 2023-09-06T15:57:41,775 adding 'colossalai/tensor/d_tensor/api.py' 2023-09-06T15:57:41,779 adding 'colossalai/tensor/d_tensor/comm_spec.py' 2023-09-06T15:57:41,782 adding 'colossalai/tensor/d_tensor/layout.py' 2023-09-06T15:57:41,788 adding 'colossalai/tensor/d_tensor/layout_converter.py' 2023-09-06T15:57:41,791 adding 'colossalai/tensor/d_tensor/misc.py' 2023-09-06T15:57:41,794 adding 'colossalai/tensor/d_tensor/sharding_spec.py' 2023-09-06T15:57:41,797 adding 'colossalai/tensor/d_tensor/utils.py' 2023-09-06T15:57:41,801 adding 'colossalai/testing/__init__.py' 2023-09-06T15:57:41,804 adding 'colossalai/testing/comparison.py' 2023-09-06T15:57:41,807 adding 'colossalai/testing/pytest_wrapper.py' 2023-09-06T15:57:41,809 adding 'colossalai/testing/random.py' 2023-09-06T15:57:41,813 adding 'colossalai/testing/utils.py' 2023-09-06T15:57:41,817 adding 'colossalai/utils/__init__.py' 2023-09-06T15:57:41,821 adding 'colossalai/utils/activation_checkpoint.py' 2023-09-06T15:57:41,825 adding 'colossalai/utils/checkpointing.py' 2023-09-06T15:57:41,831 adding 'colossalai/utils/common.py' 2023-09-06T15:57:41,834 adding 'colossalai/utils/cuda.py' 2023-09-06T15:57:41,837 adding 'colossalai/utils/memory.py' 2023-09-06T15:57:41,840 adding 'colossalai/utils/moe.py' 2023-09-06T15:57:41,843 adding 'colossalai/utils/timer.py' 2023-09-06T15:57:41,846 adding 'colossalai/utils/checkpoint/__init__.py' 2023-09-06T15:57:41,849 adding 'colossalai/utils/checkpoint/module_checkpoint.py' 2023-09-06T15:57:41,852 adding 'colossalai/utils/checkpoint/utils.py' 2023-09-06T15:57:41,856 adding 'colossalai/utils/data_sampler/__init__.py' 2023-09-06T15:57:41,858 adding 'colossalai/utils/data_sampler/base_sampler.py' 2023-09-06T15:57:41,862 adding 'colossalai/utils/data_sampler/data_parallel_sampler.py' 2023-09-06T15:57:41,865 adding 'colossalai/utils/model/__init__.py' 2023-09-06T15:57:41,868 adding 'colossalai/utils/model/utils.py' 2023-09-06T15:57:41,871 adding 'colossalai/utils/multi_tensor_apply/__init__.py' 2023-09-06T15:57:41,874 adding 'colossalai/utils/multi_tensor_apply/multi_tensor_apply.py' 2023-09-06T15:57:41,877 adding 'colossalai/utils/profiler/__init__.py' 2023-09-06T15:57:41,880 adding 'colossalai/utils/profiler/extention.py' 2023-09-06T15:57:41,884 adding 'colossalai/utils/profiler/profiler.py' 2023-09-06T15:57:41,887 adding 'colossalai/utils/profiler/stateful_tensor_mem_extention.py' 2023-09-06T15:57:41,890 adding 'colossalai/utils/profiler/legacy/__init__.py' 2023-09-06T15:57:41,894 adding 'colossalai/utils/profiler/legacy/comm_profiler.py' 2023-09-06T15:57:41,897 adding 'colossalai/utils/profiler/legacy/pcie_profiler.py' 2023-09-06T15:57:41,900 adding 'colossalai/utils/profiler/legacy/prof_utils.py' 2023-09-06T15:57:41,904 adding 'colossalai/utils/rank_recorder/__init__.py' 2023-09-06T15:57:41,907 adding 'colossalai/utils/rank_recorder/rank_recorder.py' 2023-09-06T15:57:41,911 adding 'colossalai/utils/tensor_detector/__init__.py' 2023-09-06T15:57:41,914 adding 'colossalai/utils/tensor_detector/tensor_detector.py' 2023-09-06T15:57:41,918 adding 'colossalai/zero/__init__.py' 2023-09-06T15:57:41,921 adding 'colossalai/zero/wrapper.py' 2023-09-06T15:57:41,925 adding 'colossalai/zero/gemini/__init__.py' 2023-09-06T15:57:41,929 adding 'colossalai/zero/gemini/colo_init_context.py' 2023-09-06T15:57:41,939 adding 'colossalai/zero/gemini/gemini_ddp.py' 2023-09-06T15:57:41,942 adding 'colossalai/zero/gemini/gemini_hook.py' 2023-09-06T15:57:41,946 adding 'colossalai/zero/gemini/gemini_mgr.py' 2023-09-06T15:57:41,955 adding 'colossalai/zero/gemini/gemini_optimizer.py' 2023-09-06T15:57:41,960 adding 'colossalai/zero/gemini/placement_policy.py' 2023-09-06T15:57:41,963 adding 'colossalai/zero/gemini/utils.py' 2023-09-06T15:57:41,966 adding 'colossalai/zero/gemini/chunk/__init__.py' 2023-09-06T15:57:41,972 adding 'colossalai/zero/gemini/chunk/chunk.py' 2023-09-06T15:57:41,977 adding 'colossalai/zero/gemini/chunk/manager.py' 2023-09-06T15:57:41,981 adding 'colossalai/zero/gemini/chunk/search_utils.py' 2023-09-06T15:57:41,983 adding 'colossalai/zero/gemini/chunk/utils.py' 2023-09-06T15:57:41,987 adding 'colossalai/zero/gemini/memory_tracer/__init__.py' 2023-09-06T15:57:41,990 adding 'colossalai/zero/gemini/memory_tracer/chunk_memstats_collector.py' 2023-09-06T15:57:41,993 adding 'colossalai/zero/gemini/memory_tracer/memory_monitor.py' 2023-09-06T15:57:41,995 adding 'colossalai/zero/gemini/memory_tracer/memory_stats.py' 2023-09-06T15:57:41,999 adding 'colossalai/zero/gemini/memory_tracer/memstats_collector.py' 2023-09-06T15:57:42,001 adding 'colossalai/zero/gemini/memory_tracer/param_runtime_order.py' 2023-09-06T15:57:42,004 adding 'colossalai/zero/gemini/memory_tracer/runtime_mem_tracer.py' 2023-09-06T15:57:42,007 adding 'colossalai/zero/gemini/memory_tracer/static_memstats_collector.py' 2023-09-06T15:57:42,010 adding 'colossalai/zero/gemini/memory_tracer/utils.py' 2023-09-06T15:57:42,014 adding 'colossalai/zero/legacy/__init__.py' 2023-09-06T15:57:42,018 adding 'colossalai/zero/legacy/gemini/__init__.py' 2023-09-06T15:57:42,020 adding 'colossalai/zero/legacy/gemini/gemini_context.py' 2023-09-06T15:57:42,024 adding 'colossalai/zero/legacy/gemini/stateful_tensor.py' 2023-09-06T15:57:42,027 adding 'colossalai/zero/legacy/gemini/stateful_tensor_mgr.py' 2023-09-06T15:57:42,030 adding 'colossalai/zero/legacy/gemini/tensor_placement_policy.py' 2023-09-06T15:57:42,033 adding 'colossalai/zero/legacy/gemini/tensor_utils.py' 2023-09-06T15:57:42,037 adding 'colossalai/zero/legacy/gemini/ophooks/__init__.py' 2023-09-06T15:57:42,039 adding 'colossalai/zero/legacy/gemini/ophooks/_shard_grad_ophook.py' 2023-09-06T15:57:42,042 adding 'colossalai/zero/legacy/gemini/ophooks/_shard_param_ophook.py' 2023-09-06T15:57:42,045 adding 'colossalai/zero/legacy/gemini/ophooks/runtime_mem_tracer_hook.py' 2023-09-06T15:57:42,048 adding 'colossalai/zero/legacy/gemini/ophooks/utils.py' 2023-09-06T15:57:42,051 adding 'colossalai/zero/legacy/gemini/paramhooks/__init__.py' 2023-09-06T15:57:42,054 adding 'colossalai/zero/legacy/gemini/paramhooks/_param_hookmgr.py' 2023-09-06T15:57:42,058 adding 'colossalai/zero/legacy/init_ctx/__init__.py' 2023-09-06T15:57:42,062 adding 'colossalai/zero/legacy/init_ctx/init_context.py' 2023-09-06T15:57:42,065 adding 'colossalai/zero/legacy/shard_utils/__init__.py' 2023-09-06T15:57:42,068 adding 'colossalai/zero/legacy/shard_utils/base_shard_strategy.py' 2023-09-06T15:57:42,071 adding 'colossalai/zero/legacy/shard_utils/bucket_tensor_shard_strategy.py' 2023-09-06T15:57:42,073 adding 'colossalai/zero/legacy/shard_utils/commons.py' 2023-09-06T15:57:42,076 adding 'colossalai/zero/legacy/shard_utils/tensor_shard_strategy.py' 2023-09-06T15:57:42,080 adding 'colossalai/zero/legacy/sharded_model/__init__.py' 2023-09-06T15:57:42,083 adding 'colossalai/zero/legacy/sharded_model/_utils.py' 2023-09-06T15:57:42,087 adding 'colossalai/zero/legacy/sharded_model/reduce_scatter.py' 2023-09-06T15:57:42,096 adding 'colossalai/zero/legacy/sharded_model/sharded_model_v2.py' 2023-09-06T15:57:42,099 adding 'colossalai/zero/legacy/sharded_model/utils.py' 2023-09-06T15:57:42,102 adding 'colossalai/zero/legacy/sharded_model/zero_hook.py' 2023-09-06T15:57:42,106 adding 'colossalai/zero/legacy/sharded_optim/__init__.py' 2023-09-06T15:57:42,112 adding 'colossalai/zero/legacy/sharded_optim/sharded_optim_v2.py' 2023-09-06T15:57:42,115 adding 'colossalai/zero/legacy/sharded_param/__init__.py' 2023-09-06T15:57:42,118 adding 'colossalai/zero/legacy/sharded_param/sharded_param.py' 2023-09-06T15:57:42,121 adding 'colossalai/zero/legacy/sharded_param/sharded_tensor.py' 2023-09-06T15:57:42,124 adding 'colossalai/zero/low_level/__init__.py' 2023-09-06T15:57:42,128 adding 'colossalai/zero/low_level/_utils.py' 2023-09-06T15:57:42,137 adding 'colossalai/zero/low_level/low_level_optim.py' 2023-09-06T15:57:42,141 adding 'colossalai/zero/low_level/bookkeeping/__init__.py' 2023-09-06T15:57:42,144 adding 'colossalai/zero/low_level/bookkeeping/base_store.py' 2023-09-06T15:57:42,147 adding 'colossalai/zero/low_level/bookkeeping/bucket_store.py' 2023-09-06T15:57:42,150 adding 'colossalai/zero/low_level/bookkeeping/gradient_store.py' 2023-09-06T15:57:42,152 adding 'colossalai/zero/low_level/bookkeeping/parameter_store.py' 2023-09-06T15:57:42,155 adding 'colossalai/zero/low_level/bookkeeping/tensor_bucket.py' 2023-09-06T15:57:42,164 adding 'colossalai-0.3.2.dist-info/LICENSE' 2023-09-06T15:57:42,171 adding 'colossalai-0.3.2.dist-info/METADATA' 2023-09-06T15:57:42,173 adding 'colossalai-0.3.2.dist-info/WHEEL' 2023-09-06T15:57:42,175 adding 'colossalai-0.3.2.dist-info/entry_points.txt' 2023-09-06T15:57:42,177 adding 'colossalai-0.3.2.dist-info/top_level.txt' 2023-09-06T15:57:42,200 adding 'colossalai-0.3.2.dist-info/RECORD' 2023-09-06T15:57:42,267 removing build/bdist.linux-armv7l/wheel 2023-09-06T15:57:42,853 Building wheel for colossalai (setup.py): finished with status 'done' 2023-09-06T15:57:42,895 Created wheel for colossalai: filename=colossalai-0.3.2-py3-none-any.whl size=1158212 sha256=027b105344a1cb627d6b23e1f60b88391fa3a61ede5e4ee43bed11c1bbb57c02 2023-09-06T15:57:42,897 Stored in directory: /tmp/pip-ephem-wheel-cache-2w7bmwd7/wheels/56/73/c4/bb4a303c3f2f2669605269b10ec58a00d2a97f0df237ebc7c6 2023-09-06T15:57:42,989 Successfully built colossalai 2023-09-06T15:57:43,087 Removed build tracker: '/tmp/pip-build-tracker-4h7t43l0'