2023-03-10T02:00:39,601 Created temporary directory: /tmp/pip-ephem-wheel-cache-d69qc4ai 2023-03-10T02:00:39,606 Created temporary directory: /tmp/pip-build-tracker-5zp_mdz_ 2023-03-10T02:00:39,607 Initialized build tracking at /tmp/pip-build-tracker-5zp_mdz_ 2023-03-10T02:00:39,607 Created build tracker: /tmp/pip-build-tracker-5zp_mdz_ 2023-03-10T02:00:39,608 Entered build tracker: /tmp/pip-build-tracker-5zp_mdz_ 2023-03-10T02:00:39,609 Created temporary directory: /tmp/pip-wheel-kgzc_43g 2023-03-10T02:00:39,619 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2023-03-10T02:00:39,625 Created temporary directory: /tmp/pip-ephem-wheel-cache-_js2otsl 2023-03-10T02:00:39,677 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2023-03-10T02:00:39,685 2 location(s) to search for versions of colossalai: 2023-03-10T02:00:39,685 * https://pypi.org/simple/colossalai/ 2023-03-10T02:00:39,685 * https://www.piwheels.org/simple/colossalai/ 2023-03-10T02:00:39,686 Fetching project page and analyzing links: https://pypi.org/simple/colossalai/ 2023-03-10T02:00:39,687 Getting page https://pypi.org/simple/colossalai/ 2023-03-10T02:00:39,691 Found index url https://pypi.org/simple 2023-03-10T02:00:39,880 Fetched page https://pypi.org/simple/colossalai/ as application/vnd.pypi.simple.v1+json 2023-03-10T02:00:39,886 Found link https://files.pythonhosted.org/packages/9c/c7/427dd2ed3769c6beb05cefdc18f8fad634673ed1472d5409afbdb768c7c3/colossalai-0.2.0.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.0 2023-03-10T02:00:39,888 Found link https://files.pythonhosted.org/packages/b1/70/cc6165ba3ce239c4b0c53c8ea6cabbfb31aa4b66dc580e85ff16d787384d/colossalai-0.2.1.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.1 2023-03-10T02:00:39,889 Found link https://files.pythonhosted.org/packages/87/d0/d82b49b722315654f6773eb458b929dba9949044a6b7cd087027c2391169/colossalai-0.2.2.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.2 2023-03-10T02:00:39,891 Found link https://files.pythonhosted.org/packages/c9/bb/9ec93ed622207b589fe4b1eddad9da74174eedc40ac306bff77ba823db92/colossalai-0.2.3.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.3 2023-03-10T02:00:39,892 Found link https://files.pythonhosted.org/packages/2c/26/4c38cbad383dfae44b9ececdea8f71d1be32c0ce80aadf883f915dd4d2fb/colossalai-0.2.4.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.4 2023-03-10T02:00:39,894 Found link https://files.pythonhosted.org/packages/15/05/38488d4a76c1c010da8d07e07ba14b15191d0e7398057a683f34e3c54dfe/colossalai-0.2.5.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.5 2023-03-10T02:00:39,895 Found link https://files.pythonhosted.org/packages/5a/a3/edf03a61814ca761d3133fae4bad80473c0f04693ef4af083787e71b6ac9/colossalai-0.2.6.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.6 2023-03-10T02:00:39,896 Fetching project page and analyzing links: https://www.piwheels.org/simple/colossalai/ 2023-03-10T02:00:39,897 Getting page https://www.piwheels.org/simple/colossalai/ 2023-03-10T02:00:39,899 Found index url https://www.piwheels.org/simple 2023-03-10T02:00:40,159 Fetched page https://www.piwheels.org/simple/colossalai/ as text/html 2023-03-10T02:00:40,166 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.5-py3-none-any.whl#sha256=645c27a6955e81220cbf83dbc4eec810b1b271bdbd2a695bd3f1a179a2a348ff (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T02:00:40,167 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.4-py3-none-any.whl#sha256=4c6554305658e978bfa9b199fd59b284ba6fb60cf9b17002bed4982c13f17c1f (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T02:00:40,167 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.3-py3-none-any.whl#sha256=e5c1d7c77bd0eb9c0b3cea5c33050a759bb29fcd128ef0f6459bec9e69f4969c (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T02:00:40,168 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.2-py3-none-any.whl#sha256=272ef339dfcdc089fc4e150a9ed2377e0998b1b3e5061b8760add23228609122 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T02:00:40,169 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.1-py3-none-any.whl#sha256=fba9ebaec6b5f43d7e2424a858cae18f765cf95d36956496e4476f23f36d97b0 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T02:00:40,169 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.0-py3-none-any.whl#sha256=707a78c4c8d4b96171adc704e95bbdc387fc1dee4f612a8500f0a562c4454dc7 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T02:00:40,170 Skipping link: not a file: https://www.piwheels.org/simple/colossalai/ 2023-03-10T02:00:40,170 Skipping link: not a file: https://pypi.org/simple/colossalai/ 2023-03-10T02:00:40,202 Given no hashes to check 1 links for project 'colossalai': discarding no candidates 2023-03-10T02:00:40,230 Collecting colossalai==0.2.6 2023-03-10T02:00:40,234 Created temporary directory: /tmp/pip-unpack-y7mjeo_e 2023-03-10T02:00:40,667 Downloading colossalai-0.2.6.tar.gz (681 kB) 2023-03-10T02:00:43,866 Added colossalai==0.2.6 from https://files.pythonhosted.org/packages/5a/a3/edf03a61814ca761d3133fae4bad80473c0f04693ef4af083787e71b6ac9/colossalai-0.2.6.tar.gz to build tracker '/tmp/pip-build-tracker-5zp_mdz_' 2023-03-10T02:00:43,869 Running setup.py (path:/tmp/pip-wheel-kgzc_43g/colossalai_8c80d770786f4ed78b652713057117a5/setup.py) egg_info for package colossalai 2023-03-10T02:00:43,870 Created temporary directory: /tmp/pip-pip-egg-info-cxrb1bwr 2023-03-10T02:00:43,871 Preparing metadata (setup.py): started 2023-03-10T02:00:43,873 Running command python setup.py egg_info 2023-03-10T02:00:45,521 running egg_info 2023-03-10T02:00:45,524 creating /tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info 2023-03-10T02:00:45,575 writing /tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/PKG-INFO 2023-03-10T02:00:45,581 writing dependency_links to /tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/dependency_links.txt 2023-03-10T02:00:45,584 writing entry points to /tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/entry_points.txt 2023-03-10T02:00:45,587 writing requirements to /tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/requires.txt 2023-03-10T02:00:45,590 writing top-level names to /tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/top_level.txt 2023-03-10T02:00:45,592 writing manifest file '/tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/SOURCES.txt' 2023-03-10T02:00:45,955 reading manifest file '/tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/SOURCES.txt' 2023-03-10T02:00:45,959 reading manifest template 'MANIFEST.in' 2023-03-10T02:00:47,930 warning: no files found matching '*.tr' under directory 'colossalai' 2023-03-10T02:00:48,908 warning: no files found matching '*.cc' under directory 'colossalai' 2023-03-10T02:00:49,402 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-03-10T02:00:49,421 adding license file 'LICENSE' 2023-03-10T02:00:49,489 writing manifest file '/tmp/pip-pip-egg-info-cxrb1bwr/colossalai.egg-info/SOURCES.txt' 2023-03-10T02:00:49,609 Preparing metadata (setup.py): finished with status 'done' 2023-03-10T02:00:49,626 Source in /tmp/pip-wheel-kgzc_43g/colossalai_8c80d770786f4ed78b652713057117a5 has version 0.2.6, which satisfies requirement colossalai==0.2.6 from https://files.pythonhosted.org/packages/5a/a3/edf03a61814ca761d3133fae4bad80473c0f04693ef4af083787e71b6ac9/colossalai-0.2.6.tar.gz 2023-03-10T02:00:49,628 Removed colossalai==0.2.6 from https://files.pythonhosted.org/packages/5a/a3/edf03a61814ca761d3133fae4bad80473c0f04693ef4af083787e71b6ac9/colossalai-0.2.6.tar.gz from build tracker '/tmp/pip-build-tracker-5zp_mdz_' 2023-03-10T02:00:49,646 Created temporary directory: /tmp/pip-unpack-lawv5d2t 2023-03-10T02:00:49,648 Building wheels for collected packages: colossalai 2023-03-10T02:00:49,657 Created temporary directory: /tmp/pip-wheel-tu37a4hr 2023-03-10T02:00:49,658 Building wheel for colossalai (setup.py): started 2023-03-10T02:00:49,660 Destination directory: /tmp/pip-wheel-tu37a4hr 2023-03-10T02:00:49,660 Running command python setup.py bdist_wheel 2023-03-10T02:00:51,651 running bdist_wheel 2023-03-10T02:00:51,808 running build 2023-03-10T02:00:51,808 running build_py 2023-03-10T02:00:51,866 creating build 2023-03-10T02:00:51,867 creating build/lib 2023-03-10T02:00:51,868 creating build/lib/colossalai 2023-03-10T02:00:51,870 copying colossalai/global_variables.py -> build/lib/colossalai 2023-03-10T02:00:51,874 copying colossalai/version.py -> build/lib/colossalai 2023-03-10T02:00:51,877 copying colossalai/core.py -> build/lib/colossalai 2023-03-10T02:00:51,880 copying colossalai/initialize.py -> build/lib/colossalai 2023-03-10T02:00:51,885 copying colossalai/constants.py -> build/lib/colossalai 2023-03-10T02:00:51,889 copying colossalai/__init__.py -> build/lib/colossalai 2023-03-10T02:00:51,894 creating build/lib/op_builder 2023-03-10T02:00:51,895 copying op_builder/layernorm.py -> build/lib/op_builder 2023-03-10T02:00:51,899 copying op_builder/moe.py -> build/lib/op_builder 2023-03-10T02:00:51,901 copying op_builder/scaled_masked_softmax.py -> build/lib/op_builder 2023-03-10T02:00:51,904 copying op_builder/multi_head_attn.py -> build/lib/op_builder 2023-03-10T02:00:51,907 copying op_builder/utils.py -> build/lib/op_builder 2023-03-10T02:00:51,911 copying op_builder/__init__.py -> build/lib/op_builder 2023-03-10T02:00:51,914 copying op_builder/builder.py -> build/lib/op_builder 2023-03-10T02:00:51,917 copying op_builder/fused_optim.py -> build/lib/op_builder 2023-03-10T02:00:51,921 copying op_builder/scaled_upper_triangle_masked_softmax.py -> build/lib/op_builder 2023-03-10T02:00:51,924 copying op_builder/cpu_adam.py -> build/lib/op_builder 2023-03-10T02:00:51,929 creating build/lib/colossalai/testing 2023-03-10T02:00:51,930 copying colossalai/testing/pytest_wrapper.py -> build/lib/colossalai/testing 2023-03-10T02:00:51,934 copying colossalai/testing/utils.py -> build/lib/colossalai/testing 2023-03-10T02:00:51,938 copying colossalai/testing/__init__.py -> build/lib/colossalai/testing 2023-03-10T02:00:51,941 copying colossalai/testing/random.py -> build/lib/colossalai/testing 2023-03-10T02:00:51,945 copying colossalai/testing/comparison.py -> build/lib/colossalai/testing 2023-03-10T02:00:51,949 creating build/lib/colossalai/auto_parallel 2023-03-10T02:00:51,950 copying colossalai/auto_parallel/__init__.py -> build/lib/colossalai/auto_parallel 2023-03-10T02:00:51,955 creating build/lib/colossalai/utils 2023-03-10T02:00:51,957 copying colossalai/utils/timer.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,961 copying colossalai/utils/cuda.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,964 copying colossalai/utils/moe.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,967 copying colossalai/utils/common.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,972 copying colossalai/utils/memory.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,976 copying colossalai/utils/activation_checkpoint.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,980 copying colossalai/utils/__init__.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,984 copying colossalai/utils/checkpointing.py -> build/lib/colossalai/utils 2023-03-10T02:00:51,990 creating build/lib/colossalai/pipeline 2023-03-10T02:00:51,991 copying colossalai/pipeline/layer_spec.py -> build/lib/colossalai/pipeline 2023-03-10T02:00:51,995 copying colossalai/pipeline/pipeline_process_group.py -> build/lib/colossalai/pipeline 2023-03-10T02:00:51,999 copying colossalai/pipeline/utils.py -> build/lib/colossalai/pipeline 2023-03-10T02:00:52,003 copying colossalai/pipeline/__init__.py -> build/lib/colossalai/pipeline 2023-03-10T02:00:52,007 copying colossalai/pipeline/pipelinable.py -> build/lib/colossalai/pipeline 2023-03-10T02:00:52,012 creating build/lib/colossalai/zero 2023-03-10T02:00:52,014 copying colossalai/zero/__init__.py -> build/lib/colossalai/zero 2023-03-10T02:00:52,019 creating build/lib/colossalai/builder 2023-03-10T02:00:52,020 copying colossalai/builder/__init__.py -> build/lib/colossalai/builder 2023-03-10T02:00:52,024 copying colossalai/builder/builder.py -> build/lib/colossalai/builder 2023-03-10T02:00:52,029 creating build/lib/colossalai/fx 2023-03-10T02:00:52,031 copying colossalai/fx/proxy.py -> build/lib/colossalai/fx 2023-03-10T02:00:52,034 copying colossalai/fx/_compatibility.py -> build/lib/colossalai/fx 2023-03-10T02:00:52,037 copying colossalai/fx/__init__.py -> build/lib/colossalai/fx 2023-03-10T02:00:52,041 copying colossalai/fx/_meta_registrations.py -> build/lib/colossalai/fx 2023-03-10T02:00:52,045 copying colossalai/fx/graph_module.py -> build/lib/colossalai/fx 2023-03-10T02:00:52,051 creating build/lib/colossalai/context 2023-03-10T02:00:52,052 copying colossalai/context/singleton_meta.py -> build/lib/colossalai/context 2023-03-10T02:00:52,056 copying colossalai/context/config.py -> build/lib/colossalai/context 2023-03-10T02:00:52,059 copying colossalai/context/parallel_context.py -> build/lib/colossalai/context 2023-03-10T02:00:52,064 copying colossalai/context/moe_context.py -> build/lib/colossalai/context 2023-03-10T02:00:52,068 copying colossalai/context/parallel_mode.py -> build/lib/colossalai/context 2023-03-10T02:00:52,072 copying colossalai/context/__init__.py -> build/lib/colossalai/context 2023-03-10T02:00:52,077 creating build/lib/colossalai/tensor 2023-03-10T02:00:52,079 copying colossalai/tensor/comm_spec.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,084 copying colossalai/tensor/dist_spec_mgr.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,088 copying colossalai/tensor/const.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,092 copying colossalai/tensor/compute_spec.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,095 copying colossalai/tensor/shape_consistency.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,101 copying colossalai/tensor/sharding_spec.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,105 copying colossalai/tensor/process_group.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,109 copying colossalai/tensor/distspec.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,113 copying colossalai/tensor/op_wrapper.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,116 copying colossalai/tensor/colo_parameter.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,120 copying colossalai/tensor/utils.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,124 copying colossalai/tensor/colo_tensor.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,128 copying colossalai/tensor/__init__.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,132 copying colossalai/tensor/param_op_hook.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,136 copying colossalai/tensor/tensor_spec.py -> build/lib/colossalai/tensor 2023-03-10T02:00:52,140 creating build/lib/colossalai/kernel 2023-03-10T02:00:52,142 copying colossalai/kernel/__init__.py -> build/lib/colossalai/kernel 2023-03-10T02:00:52,147 creating build/lib/colossalai/amp 2023-03-10T02:00:52,148 copying colossalai/amp/amp_type.py -> build/lib/colossalai/amp 2023-03-10T02:00:52,152 copying colossalai/amp/__init__.py -> build/lib/colossalai/amp 2023-03-10T02:00:52,157 creating build/lib/colossalai/device 2023-03-10T02:00:52,159 copying colossalai/device/device_mesh.py -> build/lib/colossalai/device 2023-03-10T02:00:52,163 copying colossalai/device/calc_pipeline_strategy.py -> build/lib/colossalai/device 2023-03-10T02:00:52,167 copying colossalai/device/__init__.py -> build/lib/colossalai/device 2023-03-10T02:00:52,170 copying colossalai/device/alpha_beta_profiler.py -> build/lib/colossalai/device 2023-03-10T02:00:52,177 creating build/lib/colossalai/gemini 2023-03-10T02:00:52,178 copying colossalai/gemini/stateful_tensor_mgr.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,182 copying colossalai/gemini/tensor_placement_policy.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,186 copying colossalai/gemini/gemini_context.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,190 copying colossalai/gemini/gemini_mgr.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,194 copying colossalai/gemini/placement_policy.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,198 copying colossalai/gemini/__init__.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,201 copying colossalai/gemini/tensor_utils.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,205 copying colossalai/gemini/stateful_tensor.py -> build/lib/colossalai/gemini 2023-03-10T02:00:52,210 creating build/lib/colossalai/logging 2023-03-10T02:00:52,212 copying colossalai/logging/logger.py -> build/lib/colossalai/logging 2023-03-10T02:00:52,216 copying colossalai/logging/__init__.py -> build/lib/colossalai/logging 2023-03-10T02:00:52,220 creating build/lib/colossalai/_C 2023-03-10T02:00:52,222 copying colossalai/_C/__init__.py -> build/lib/colossalai/_C 2023-03-10T02:00:52,226 creating build/lib/colossalai/booster 2023-03-10T02:00:52,228 copying colossalai/booster/accelerator.py -> build/lib/colossalai/booster 2023-03-10T02:00:52,231 copying colossalai/booster/plugin.py -> build/lib/colossalai/booster 2023-03-10T02:00:52,235 copying colossalai/booster/booster.py -> build/lib/colossalai/booster 2023-03-10T02:00:52,238 copying colossalai/booster/precision.py -> build/lib/colossalai/booster 2023-03-10T02:00:52,241 copying colossalai/booster/__init__.py -> build/lib/colossalai/booster 2023-03-10T02:00:52,244 copying colossalai/booster/environment_table.py -> build/lib/colossalai/booster 2023-03-10T02:00:52,250 creating build/lib/colossalai/engine 2023-03-10T02:00:52,251 copying colossalai/engine/_base_engine.py -> build/lib/colossalai/engine 2023-03-10T02:00:52,255 copying colossalai/engine/__init__.py -> build/lib/colossalai/engine 2023-03-10T02:00:52,260 creating build/lib/colossalai/trainer 2023-03-10T02:00:52,261 copying colossalai/trainer/_trainer.py -> build/lib/colossalai/trainer 2023-03-10T02:00:52,266 copying colossalai/trainer/__init__.py -> build/lib/colossalai/trainer 2023-03-10T02:00:52,271 creating build/lib/colossalai/communication 2023-03-10T02:00:52,272 copying colossalai/communication/collective.py -> build/lib/colossalai/communication 2023-03-10T02:00:52,277 copying colossalai/communication/ring.py -> build/lib/colossalai/communication 2023-03-10T02:00:52,280 copying colossalai/communication/p2p.py -> build/lib/colossalai/communication 2023-03-10T02:00:52,285 copying colossalai/communication/p2p_v2.py -> build/lib/colossalai/communication 2023-03-10T02:00:52,289 copying colossalai/communication/utils.py -> build/lib/colossalai/communication 2023-03-10T02:00:52,293 copying colossalai/communication/__init__.py -> build/lib/colossalai/communication 2023-03-10T02:00:52,297 creating build/lib/colossalai/registry 2023-03-10T02:00:52,299 copying colossalai/registry/registry.py -> build/lib/colossalai/registry 2023-03-10T02:00:52,303 copying colossalai/registry/__init__.py -> build/lib/colossalai/registry 2023-03-10T02:00:52,307 creating build/lib/colossalai/nn 2023-03-10T02:00:52,309 copying colossalai/nn/init.py -> build/lib/colossalai/nn 2023-03-10T02:00:52,313 copying colossalai/nn/__init__.py -> build/lib/colossalai/nn 2023-03-10T02:00:52,317 creating build/lib/colossalai/cli 2023-03-10T02:00:52,319 copying colossalai/cli/cli.py -> build/lib/colossalai/cli 2023-03-10T02:00:52,322 copying colossalai/cli/__init__.py -> build/lib/colossalai/cli 2023-03-10T02:00:52,327 creating build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:52,328 copying colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:52,332 copying colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:52,335 copying colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:52,340 copying colossalai/auto_parallel/checkpoint/__init__.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:52,343 copying colossalai/auto_parallel/checkpoint/operation.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:52,347 copying colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:52,353 creating build/lib/colossalai/auto_parallel/passes 2023-03-10T02:00:52,355 copying colossalai/auto_parallel/passes/meta_info_prop.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T02:00:52,359 copying colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T02:00:52,364 copying colossalai/auto_parallel/passes/constants.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T02:00:52,367 copying colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T02:00:52,373 copying colossalai/auto_parallel/passes/__init__.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T02:00:52,376 copying colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T02:00:52,381 creating build/lib/colossalai/auto_parallel/pipeline_shard 2023-03-10T02:00:52,383 copying colossalai/auto_parallel/pipeline_shard/__init__.py -> build/lib/colossalai/auto_parallel/pipeline_shard 2023-03-10T02:00:52,388 creating build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:52,390 copying colossalai/auto_parallel/tensor_shard/options.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:52,395 copying colossalai/auto_parallel/tensor_shard/initialize.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:52,400 copying colossalai/auto_parallel/tensor_shard/constants.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:52,404 copying colossalai/auto_parallel/tensor_shard/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:52,406 copying colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:52,412 creating build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:52,414 copying colossalai/auto_parallel/meta_profiler/metainfo.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:52,418 copying colossalai/auto_parallel/meta_profiler/registry.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:52,422 copying colossalai/auto_parallel/meta_profiler/constants.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:52,426 copying colossalai/auto_parallel/meta_profiler/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:52,431 creating build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:52,434 copying colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:52,439 copying colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:52,443 copying colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:52,448 copying colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:52,452 copying colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:52,456 copying colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:52,463 creating build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:52,465 copying colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:52,470 copying colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:52,475 copying colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:52,479 copying colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:52,482 copying colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:52,490 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,492 copying colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,496 copying colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,500 copying colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,504 copying colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,509 copying colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,512 copying colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,516 copying colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,520 copying colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,524 copying colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,528 copying colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,532 copying colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,535 copying colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,538 copying colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,542 copying colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,545 copying colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,549 copying colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,553 copying colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,556 copying colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,560 copying colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,563 copying colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,567 copying colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,570 copying colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,574 copying colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,577 copying colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,582 copying colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,586 copying colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,590 copying colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:52,597 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,600 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,605 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,609 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,614 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,618 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,623 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,628 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,633 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,638 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,642 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,646 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,651 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,654 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,658 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,662 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,666 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,670 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,674 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,680 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:52,686 creating build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,687 copying colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,692 copying colossalai/auto_parallel/meta_profiler/meta_registry/where.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,695 copying colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,699 copying colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,702 copying colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,707 copying colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,710 copying colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,714 copying colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,718 copying colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,722 copying colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,725 copying colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:52,730 creating build/lib/colossalai/utils/model 2023-03-10T02:00:52,732 copying colossalai/utils/model/lazy_init_context.py -> build/lib/colossalai/utils/model 2023-03-10T02:00:52,736 copying colossalai/utils/model/colo_init_context.py -> build/lib/colossalai/utils/model 2023-03-10T02:00:52,740 copying colossalai/utils/model/experimental.py -> build/lib/colossalai/utils/model 2023-03-10T02:00:52,745 copying colossalai/utils/model/utils.py -> build/lib/colossalai/utils/model 2023-03-10T02:00:52,748 copying colossalai/utils/model/__init__.py -> build/lib/colossalai/utils/model 2023-03-10T02:00:52,752 creating build/lib/colossalai/utils/rank_recorder 2023-03-10T02:00:52,753 copying colossalai/utils/rank_recorder/__init__.py -> build/lib/colossalai/utils/rank_recorder 2023-03-10T02:00:52,757 copying colossalai/utils/rank_recorder/rank_recorder.py -> build/lib/colossalai/utils/rank_recorder 2023-03-10T02:00:52,762 creating build/lib/colossalai/utils/multi_tensor_apply 2023-03-10T02:00:52,764 copying colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-03-10T02:00:52,767 copying colossalai/utils/multi_tensor_apply/__init__.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-03-10T02:00:52,772 creating build/lib/colossalai/utils/checkpoint 2023-03-10T02:00:52,774 copying colossalai/utils/checkpoint/module_checkpoint.py -> build/lib/colossalai/utils/checkpoint 2023-03-10T02:00:52,779 copying colossalai/utils/checkpoint/utils.py -> build/lib/colossalai/utils/checkpoint 2023-03-10T02:00:52,783 copying colossalai/utils/checkpoint/__init__.py -> build/lib/colossalai/utils/checkpoint 2023-03-10T02:00:52,787 creating build/lib/colossalai/utils/profiler 2023-03-10T02:00:52,789 copying colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/lib/colossalai/utils/profiler 2023-03-10T02:00:52,793 copying colossalai/utils/profiler/profiler.py -> build/lib/colossalai/utils/profiler 2023-03-10T02:00:52,797 copying colossalai/utils/profiler/__init__.py -> build/lib/colossalai/utils/profiler 2023-03-10T02:00:52,801 copying colossalai/utils/profiler/extention.py -> build/lib/colossalai/utils/profiler 2023-03-10T02:00:52,806 creating build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,808 copying colossalai/utils/checkpoint_io/convertor.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,813 copying colossalai/utils/checkpoint_io/reader.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,817 copying colossalai/utils/checkpoint_io/io.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,821 copying colossalai/utils/checkpoint_io/meta.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,824 copying colossalai/utils/checkpoint_io/distributed.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,829 copying colossalai/utils/checkpoint_io/utils.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,833 copying colossalai/utils/checkpoint_io/constant.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,836 copying colossalai/utils/checkpoint_io/backend.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,840 copying colossalai/utils/checkpoint_io/__init__.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,843 copying colossalai/utils/checkpoint_io/writer.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T02:00:52,848 creating build/lib/colossalai/utils/tensor_detector 2023-03-10T02:00:52,849 copying colossalai/utils/tensor_detector/tensor_detector.py -> build/lib/colossalai/utils/tensor_detector 2023-03-10T02:00:52,854 copying colossalai/utils/tensor_detector/__init__.py -> build/lib/colossalai/utils/tensor_detector 2023-03-10T02:00:52,858 creating build/lib/colossalai/utils/data_sampler 2023-03-10T02:00:52,860 copying colossalai/utils/data_sampler/base_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-03-10T02:00:52,863 copying colossalai/utils/data_sampler/__init__.py -> build/lib/colossalai/utils/data_sampler 2023-03-10T02:00:52,867 copying colossalai/utils/data_sampler/data_parallel_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-03-10T02:00:52,873 creating build/lib/colossalai/utils/profiler/legacy 2023-03-10T02:00:52,875 copying colossalai/utils/profiler/legacy/pcie_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T02:00:52,879 copying colossalai/utils/profiler/legacy/prof_utils.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T02:00:52,883 copying colossalai/utils/profiler/legacy/comm_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T02:00:52,887 copying colossalai/utils/profiler/legacy/__init__.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T02:00:52,892 creating build/lib/colossalai/pipeline/rpc 2023-03-10T02:00:52,894 copying colossalai/pipeline/rpc/utils.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T02:00:52,898 copying colossalai/pipeline/rpc/_pipeline_base.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T02:00:52,905 copying colossalai/pipeline/rpc/_pipeline_schedule.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T02:00:52,910 copying colossalai/pipeline/rpc/__init__.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T02:00:52,914 creating build/lib/colossalai/pipeline/middleware 2023-03-10T02:00:52,916 copying colossalai/pipeline/middleware/topo.py -> build/lib/colossalai/pipeline/middleware 2023-03-10T02:00:52,921 copying colossalai/pipeline/middleware/__init__.py -> build/lib/colossalai/pipeline/middleware 2023-03-10T02:00:52,925 creating build/lib/colossalai/pipeline/middleware/adaptor 2023-03-10T02:00:52,927 copying colossalai/pipeline/middleware/adaptor/fx.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-03-10T02:00:52,931 copying colossalai/pipeline/middleware/adaptor/__init__.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-03-10T02:00:52,936 creating build/lib/colossalai/zero/utils 2023-03-10T02:00:52,937 copying colossalai/zero/utils/__init__.py -> build/lib/colossalai/zero/utils 2023-03-10T02:00:52,941 copying colossalai/zero/utils/gemini_hook.py -> build/lib/colossalai/zero/utils 2023-03-10T02:00:52,944 copying colossalai/zero/utils/zero_hook.py -> build/lib/colossalai/zero/utils 2023-03-10T02:00:52,950 creating build/lib/colossalai/zero/sharded_param 2023-03-10T02:00:52,951 copying colossalai/zero/sharded_param/__init__.py -> build/lib/colossalai/zero/sharded_param 2023-03-10T02:00:52,955 copying colossalai/zero/sharded_param/sharded_param.py -> build/lib/colossalai/zero/sharded_param 2023-03-10T02:00:52,958 copying colossalai/zero/sharded_param/sharded_tensor.py -> build/lib/colossalai/zero/sharded_param 2023-03-10T02:00:52,963 creating build/lib/colossalai/zero/init_ctx 2023-03-10T02:00:52,965 copying colossalai/zero/init_ctx/init_context.py -> build/lib/colossalai/zero/init_ctx 2023-03-10T02:00:52,969 copying colossalai/zero/init_ctx/__init__.py -> build/lib/colossalai/zero/init_ctx 2023-03-10T02:00:52,974 creating build/lib/colossalai/zero/sharded_optim 2023-03-10T02:00:52,976 copying colossalai/zero/sharded_optim/sharded_optim_v2.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T02:00:52,981 copying colossalai/zero/sharded_optim/low_level_optim.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T02:00:52,986 copying colossalai/zero/sharded_optim/__init__.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T02:00:52,990 copying colossalai/zero/sharded_optim/_utils.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T02:00:52,995 creating build/lib/colossalai/zero/sharded_model 2023-03-10T02:00:52,997 copying colossalai/zero/sharded_model/reduce_scatter.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T02:00:53,002 copying colossalai/zero/sharded_model/sharded_model_v2.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T02:00:53,007 copying colossalai/zero/sharded_model/utils.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T02:00:53,010 copying colossalai/zero/sharded_model/__init__.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T02:00:53,014 copying colossalai/zero/sharded_model/_utils.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T02:00:53,019 creating build/lib/colossalai/zero/shard_utils 2023-03-10T02:00:53,021 copying colossalai/zero/shard_utils/commons.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T02:00:53,024 copying colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T02:00:53,028 copying colossalai/zero/shard_utils/tensor_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T02:00:53,031 copying colossalai/zero/shard_utils/__init__.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T02:00:53,035 copying colossalai/zero/shard_utils/base_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T02:00:53,040 creating build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:53,042 copying colossalai/zero/sharded_optim/bookkeeping/bucket_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:53,046 copying colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:53,050 copying colossalai/zero/sharded_optim/bookkeeping/parameter_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:53,053 copying colossalai/zero/sharded_optim/bookkeeping/base_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:53,057 copying colossalai/zero/sharded_optim/bookkeeping/__init__.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:53,060 copying colossalai/zero/sharded_optim/bookkeeping/gradient_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:53,065 creating build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,067 copying colossalai/fx/tracer/_tracer_utils.py -> build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,071 copying colossalai/fx/tracer/_meta_trace.py -> build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,075 copying colossalai/fx/tracer/registry.py -> build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,078 copying colossalai/fx/tracer/_symbolic_trace.py -> build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,082 copying colossalai/fx/tracer/experimental.py -> build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,087 copying colossalai/fx/tracer/__init__.py -> build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,091 copying colossalai/fx/tracer/tracer.py -> build/lib/colossalai/fx/tracer 2023-03-10T02:00:53,097 creating build/lib/colossalai/fx/codegen 2023-03-10T02:00:53,099 copying colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/lib/colossalai/fx/codegen 2023-03-10T02:00:53,105 copying colossalai/fx/codegen/__init__.py -> build/lib/colossalai/fx/codegen 2023-03-10T02:00:53,111 creating build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,112 copying colossalai/fx/profiler/tensor.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,117 copying colossalai/fx/profiler/shard_utils.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,121 copying colossalai/fx/profiler/constants.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,125 copying colossalai/fx/profiler/profiler.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,130 copying colossalai/fx/profiler/memory_utils.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,133 copying colossalai/fx/profiler/__init__.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,136 copying colossalai/fx/profiler/opcount.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,141 copying colossalai/fx/profiler/dataflow.py -> build/lib/colossalai/fx/profiler 2023-03-10T02:00:53,147 creating build/lib/colossalai/fx/passes 2023-03-10T02:00:53,149 copying colossalai/fx/passes/meta_info_prop.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,154 copying colossalai/fx/passes/passes_for_gpt2_test.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,159 copying colossalai/fx/passes/concrete_info_prop.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,163 copying colossalai/fx/passes/shard_1d_pass.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,167 copying colossalai/fx/passes/utils.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,171 copying colossalai/fx/passes/split_module.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,176 copying colossalai/fx/passes/__init__.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,179 copying colossalai/fx/passes/adding_split_node_pass.py -> build/lib/colossalai/fx/passes 2023-03-10T02:00:53,185 creating build/lib/colossalai/fx/tracer/bias_addition_patch 2023-03-10T02:00:53,186 copying colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch 2023-03-10T02:00:53,191 creating build/lib/colossalai/fx/tracer/meta_patch 2023-03-10T02:00:53,193 copying colossalai/fx/tracer/meta_patch/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch 2023-03-10T02:00:53,198 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:53,200 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:53,204 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:53,207 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:53,211 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:53,214 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:53,220 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:53,222 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:53,226 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:53,229 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:53,233 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:53,238 creating build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,240 copying colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,245 copying colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,248 copying colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,252 copying colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,256 copying colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,259 copying colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,263 copying colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,267 copying colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:53,272 creating build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,274 copying colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,278 copying colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,281 copying colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,285 copying colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,288 copying colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,293 copying colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,297 copying colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,301 copying colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:53,306 creating build/lib/colossalai/fx/profiler/experimental 2023-03-10T02:00:53,308 copying colossalai/fx/profiler/experimental/registry.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T02:00:53,312 copying colossalai/fx/profiler/experimental/shard_utils.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T02:00:53,315 copying colossalai/fx/profiler/experimental/constants.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T02:00:53,319 copying colossalai/fx/profiler/experimental/profiler.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T02:00:53,323 copying colossalai/fx/profiler/experimental/__init__.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T02:00:53,330 creating build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,332 copying colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,335 copying colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,339 copying colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,343 copying colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,346 copying colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,350 copying colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,354 copying colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,357 copying colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,361 copying colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,365 copying colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,368 copying colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:53,374 creating build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,376 copying colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,380 copying colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,383 copying colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,387 copying colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,391 copying colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,395 copying colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,398 copying colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,401 copying colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,405 copying colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:53,411 creating build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,412 copying colossalai/context/process_group_initializer/initializer_tensor.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,416 copying colossalai/context/process_group_initializer/initializer_3d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,421 copying colossalai/context/process_group_initializer/initializer_2p5d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,425 copying colossalai/context/process_group_initializer/initializer_2d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,429 copying colossalai/context/process_group_initializer/initializer_sequence.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,433 copying colossalai/context/process_group_initializer/initializer_1d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,436 copying colossalai/context/process_group_initializer/initializer_pipeline.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,440 copying colossalai/context/process_group_initializer/initializer_model.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,443 copying colossalai/context/process_group_initializer/__init__.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,447 copying colossalai/context/process_group_initializer/initializer_data.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,451 copying colossalai/context/process_group_initializer/process_group_initializer.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T02:00:53,455 creating build/lib/colossalai/context/random 2023-03-10T02:00:53,457 copying colossalai/context/random/_helper.py -> build/lib/colossalai/context/random 2023-03-10T02:00:53,462 copying colossalai/context/random/__init__.py -> build/lib/colossalai/context/random 2023-03-10T02:00:53,465 copying colossalai/context/random/seed_manager.py -> build/lib/colossalai/context/random 2023-03-10T02:00:53,470 creating build/lib/colossalai/tensor/d_tensor 2023-03-10T02:00:53,472 copying colossalai/tensor/d_tensor/comm_spec.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T02:00:53,477 copying colossalai/tensor/d_tensor/sharding_spec.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T02:00:53,482 copying colossalai/tensor/d_tensor/d_tensor.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T02:00:53,486 copying colossalai/tensor/d_tensor/misc.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T02:00:53,489 copying colossalai/tensor/d_tensor/__init__.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T02:00:53,492 copying colossalai/tensor/d_tensor/layout.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T02:00:53,498 creating build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,500 copying colossalai/kernel/op_builder/layernorm.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,504 copying colossalai/kernel/op_builder/moe.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,507 copying colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,511 copying colossalai/kernel/op_builder/multi_head_attn.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,514 copying colossalai/kernel/op_builder/utils.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,518 copying colossalai/kernel/op_builder/__init__.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,522 copying colossalai/kernel/op_builder/builder.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,527 copying colossalai/kernel/op_builder/fused_optim.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,530 copying colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,534 copying colossalai/kernel/op_builder/cpu_adam.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T02:00:53,540 creating build/lib/colossalai/kernel/cuda_native 2023-03-10T02:00:53,542 copying colossalai/kernel/cuda_native/multihead_attention.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T02:00:53,546 copying colossalai/kernel/cuda_native/flash_attention.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T02:00:53,552 copying colossalai/kernel/cuda_native/layer_norm.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T02:00:53,555 copying colossalai/kernel/cuda_native/__init__.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T02:00:53,559 copying colossalai/kernel/cuda_native/scaled_softmax.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T02:00:53,564 creating build/lib/colossalai/kernel/jit 2023-03-10T02:00:53,566 copying colossalai/kernel/jit/option.py -> build/lib/colossalai/kernel/jit 2023-03-10T02:00:53,570 copying colossalai/kernel/jit/bias_dropout_add.py -> build/lib/colossalai/kernel/jit 2023-03-10T02:00:53,573 copying colossalai/kernel/jit/__init__.py -> build/lib/colossalai/kernel/jit 2023-03-10T02:00:53,577 copying colossalai/kernel/jit/bias_gelu.py -> build/lib/colossalai/kernel/jit 2023-03-10T02:00:53,582 creating build/lib/colossalai/amp/torch_amp 2023-03-10T02:00:53,584 copying colossalai/amp/torch_amp/torch_amp.py -> build/lib/colossalai/amp/torch_amp 2023-03-10T02:00:53,588 copying colossalai/amp/torch_amp/_grad_scaler.py -> build/lib/colossalai/amp/torch_amp 2023-03-10T02:00:53,593 copying colossalai/amp/torch_amp/__init__.py -> build/lib/colossalai/amp/torch_amp 2023-03-10T02:00:53,598 creating build/lib/colossalai/amp/naive_amp 2023-03-10T02:00:53,600 copying colossalai/amp/naive_amp/_fp16_optimizer.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T02:00:53,605 copying colossalai/amp/naive_amp/naive_amp.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T02:00:53,609 copying colossalai/amp/naive_amp/__init__.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T02:00:53,612 copying colossalai/amp/naive_amp/_utils.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T02:00:53,617 creating build/lib/colossalai/amp/apex_amp 2023-03-10T02:00:53,619 copying colossalai/amp/apex_amp/apex_amp.py -> build/lib/colossalai/amp/apex_amp 2023-03-10T02:00:53,623 copying colossalai/amp/apex_amp/__init__.py -> build/lib/colossalai/amp/apex_amp 2023-03-10T02:00:53,628 creating build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:53,629 copying colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:53,633 copying colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:53,636 copying colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:53,640 copying colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:53,645 creating build/lib/colossalai/gemini/ophooks 2023-03-10T02:00:53,647 copying colossalai/gemini/ophooks/_shard_param_ophook.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T02:00:53,651 copying colossalai/gemini/ophooks/_shard_grad_ophook.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T02:00:53,654 copying colossalai/gemini/ophooks/utils.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T02:00:53,658 copying colossalai/gemini/ophooks/__init__.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T02:00:53,662 copying colossalai/gemini/ophooks/runtime_mem_tracer_hook.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T02:00:53,667 creating build/lib/colossalai/gemini/chunk 2023-03-10T02:00:53,669 copying colossalai/gemini/chunk/search_utils.py -> build/lib/colossalai/gemini/chunk 2023-03-10T02:00:53,674 copying colossalai/gemini/chunk/chunk.py -> build/lib/colossalai/gemini/chunk 2023-03-10T02:00:53,679 copying colossalai/gemini/chunk/manager.py -> build/lib/colossalai/gemini/chunk 2023-03-10T02:00:53,683 copying colossalai/gemini/chunk/utils.py -> build/lib/colossalai/gemini/chunk 2023-03-10T02:00:53,686 copying colossalai/gemini/chunk/__init__.py -> build/lib/colossalai/gemini/chunk 2023-03-10T02:00:53,692 creating build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,694 copying colossalai/gemini/memory_tracer/memory_monitor.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,698 copying colossalai/gemini/memory_tracer/memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,702 copying colossalai/gemini/memory_tracer/runtime_mem_tracer.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,705 copying colossalai/gemini/memory_tracer/memory_stats.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,709 copying colossalai/gemini/memory_tracer/utils.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,713 copying colossalai/gemini/memory_tracer/param_runtime_order.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,716 copying colossalai/gemini/memory_tracer/__init__.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,720 copying colossalai/gemini/memory_tracer/chunk_memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,723 copying colossalai/gemini/memory_tracer/static_memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T02:00:53,728 creating build/lib/colossalai/gemini/paramhooks 2023-03-10T02:00:53,730 copying colossalai/gemini/paramhooks/_param_hookmgr.py -> build/lib/colossalai/gemini/paramhooks 2023-03-10T02:00:53,734 copying colossalai/gemini/paramhooks/__init__.py -> build/lib/colossalai/gemini/paramhooks 2023-03-10T02:00:53,739 creating build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,741 copying colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,745 copying colossalai/engine/gradient_handler/_base_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,748 copying colossalai/engine/gradient_handler/utils.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,751 copying colossalai/engine/gradient_handler/__init__.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,755 copying colossalai/engine/gradient_handler/_zero_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,758 copying colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,762 copying colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,765 copying colossalai/engine/gradient_handler/_moe_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T02:00:53,771 creating build/lib/colossalai/engine/schedule 2023-03-10T02:00:53,772 copying colossalai/engine/schedule/_base_schedule.py -> build/lib/colossalai/engine/schedule 2023-03-10T02:00:53,777 copying colossalai/engine/schedule/_non_pipeline_schedule.py -> build/lib/colossalai/engine/schedule 2023-03-10T02:00:53,781 copying colossalai/engine/schedule/_pipeline_schedule_v2.py -> build/lib/colossalai/engine/schedule 2023-03-10T02:00:53,785 copying colossalai/engine/schedule/_pipeline_schedule.py -> build/lib/colossalai/engine/schedule 2023-03-10T02:00:53,792 copying colossalai/engine/schedule/__init__.py -> build/lib/colossalai/engine/schedule 2023-03-10T02:00:53,797 creating build/lib/colossalai/engine/gradient_accumulation 2023-03-10T02:00:53,799 copying colossalai/engine/gradient_accumulation/__init__.py -> build/lib/colossalai/engine/gradient_accumulation 2023-03-10T02:00:53,803 copying colossalai/engine/gradient_accumulation/_gradient_accumulation.py -> build/lib/colossalai/engine/gradient_accumulation 2023-03-10T02:00:53,809 creating build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,811 copying colossalai/trainer/hooks/_base_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,815 copying colossalai/trainer/hooks/_lr_scheduler_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,818 copying colossalai/trainer/hooks/_checkpoint_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,822 copying colossalai/trainer/hooks/_commons_.py -> build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,825 copying colossalai/trainer/hooks/_log_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,830 copying colossalai/trainer/hooks/_metric_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,834 copying colossalai/trainer/hooks/__init__.py -> build/lib/colossalai/trainer/hooks 2023-03-10T02:00:53,840 creating build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,842 copying colossalai/nn/optimizer/gemini_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,846 copying colossalai/nn/optimizer/lars.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,849 copying colossalai/nn/optimizer/hybrid_adam.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,854 copying colossalai/nn/optimizer/fused_sgd.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,858 copying colossalai/nn/optimizer/zero_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,862 copying colossalai/nn/optimizer/colossalai_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,866 copying colossalai/nn/optimizer/fused_lamb.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,870 copying colossalai/nn/optimizer/nvme_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,874 copying colossalai/nn/optimizer/lamb.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,878 copying colossalai/nn/optimizer/__init__.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,881 copying colossalai/nn/optimizer/fused_adam.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,885 copying colossalai/nn/optimizer/cpu_adam.py -> build/lib/colossalai/nn/optimizer 2023-03-10T02:00:53,891 creating build/lib/colossalai/nn/metric 2023-03-10T02:00:53,893 copying colossalai/nn/metric/accuracy_3d.py -> build/lib/colossalai/nn/metric 2023-03-10T02:00:53,896 copying colossalai/nn/metric/accuracy_2d.py -> build/lib/colossalai/nn/metric 2023-03-10T02:00:53,900 copying colossalai/nn/metric/accuracy_2p5d.py -> build/lib/colossalai/nn/metric 2023-03-10T02:00:53,903 copying colossalai/nn/metric/__init__.py -> build/lib/colossalai/nn/metric 2023-03-10T02:00:53,907 copying colossalai/nn/metric/_utils.py -> build/lib/colossalai/nn/metric 2023-03-10T02:00:53,912 creating build/lib/colossalai/nn/loss 2023-03-10T02:00:53,914 copying colossalai/nn/loss/loss_3d.py -> build/lib/colossalai/nn/loss 2023-03-10T02:00:53,918 copying colossalai/nn/loss/loss_1d.py -> build/lib/colossalai/nn/loss 2023-03-10T02:00:53,922 copying colossalai/nn/loss/loss_moe.py -> build/lib/colossalai/nn/loss 2023-03-10T02:00:53,926 copying colossalai/nn/loss/loss_2p5d.py -> build/lib/colossalai/nn/loss 2023-03-10T02:00:53,930 copying colossalai/nn/loss/__init__.py -> build/lib/colossalai/nn/loss 2023-03-10T02:00:53,933 copying colossalai/nn/loss/loss_2d.py -> build/lib/colossalai/nn/loss 2023-03-10T02:00:53,939 creating build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,941 copying colossalai/nn/_ops/addmm.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,945 copying colossalai/nn/_ops/layernorm.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,949 copying colossalai/nn/_ops/batch_norm.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,952 copying colossalai/nn/_ops/loss.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,956 copying colossalai/nn/_ops/linear.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,960 copying colossalai/nn/_ops/embedding.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,964 copying colossalai/nn/_ops/__init__.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,968 copying colossalai/nn/_ops/element_wise.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,973 copying colossalai/nn/_ops/view.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,976 copying colossalai/nn/_ops/embedding_bag.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,980 copying colossalai/nn/_ops/_utils.py -> build/lib/colossalai/nn/_ops 2023-03-10T02:00:53,987 creating build/lib/colossalai/nn/parallel 2023-03-10T02:00:53,989 copying colossalai/nn/parallel/zero_wrapper.py -> build/lib/colossalai/nn/parallel 2023-03-10T02:00:53,993 copying colossalai/nn/parallel/gemini_parallel.py -> build/lib/colossalai/nn/parallel 2023-03-10T02:00:53,997 copying colossalai/nn/parallel/reducer.py -> build/lib/colossalai/nn/parallel 2023-03-10T02:00:54,000 copying colossalai/nn/parallel/utils.py -> build/lib/colossalai/nn/parallel 2023-03-10T02:00:54,004 copying colossalai/nn/parallel/__init__.py -> build/lib/colossalai/nn/parallel 2023-03-10T02:00:54,008 copying colossalai/nn/parallel/data_parallel.py -> build/lib/colossalai/nn/parallel 2023-03-10T02:00:54,015 creating build/lib/colossalai/nn/layer 2023-03-10T02:00:54,017 copying colossalai/nn/layer/base_layer.py -> build/lib/colossalai/nn/layer 2023-03-10T02:00:54,021 copying colossalai/nn/layer/__init__.py -> build/lib/colossalai/nn/layer 2023-03-10T02:00:54,026 creating build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,028 copying colossalai/nn/lr_scheduler/multistep.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,032 copying colossalai/nn/lr_scheduler/poly.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,036 copying colossalai/nn/lr_scheduler/linear.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,040 copying colossalai/nn/lr_scheduler/torch.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,043 copying colossalai/nn/lr_scheduler/cosine.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,047 copying colossalai/nn/lr_scheduler/__init__.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,051 copying colossalai/nn/lr_scheduler/onecycle.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,055 copying colossalai/nn/lr_scheduler/delayed.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T02:00:54,061 creating build/lib/colossalai/nn/parallel/layers 2023-03-10T02:00:54,063 copying colossalai/nn/parallel/layers/module_utils.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T02:00:54,068 copying colossalai/nn/parallel/layers/linear.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T02:00:54,072 copying colossalai/nn/parallel/layers/colo_module.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T02:00:54,075 copying colossalai/nn/parallel/layers/embedding.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T02:00:54,079 copying colossalai/nn/parallel/layers/__init__.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T02:00:54,085 creating build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,087 copying colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,091 copying colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,095 copying colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,098 copying colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,121 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,126 copying colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,132 copying colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,136 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,141 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:54,146 creating build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:54,149 copying colossalai/nn/layer/parallel_3d/_operation.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:54,155 copying colossalai/nn/layer/parallel_3d/layers.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:54,163 copying colossalai/nn/layer/parallel_3d/__init__.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:54,166 copying colossalai/nn/layer/parallel_3d/_utils.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:54,171 creating build/lib/colossalai/nn/layer/utils 2023-03-10T02:00:54,173 copying colossalai/nn/layer/utils/common.py -> build/lib/colossalai/nn/layer/utils 2023-03-10T02:00:54,177 copying colossalai/nn/layer/utils/__init__.py -> build/lib/colossalai/nn/layer/utils 2023-03-10T02:00:54,182 creating build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:54,184 copying colossalai/nn/layer/parallel_1d/_operation.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:54,188 copying colossalai/nn/layer/parallel_1d/layers.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:54,195 copying colossalai/nn/layer/parallel_1d/__init__.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:54,198 copying colossalai/nn/layer/parallel_1d/_utils.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:54,204 creating build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:54,206 copying colossalai/nn/layer/parallel_sequence/_operation.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:54,211 copying colossalai/nn/layer/parallel_sequence/layers.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:54,215 copying colossalai/nn/layer/parallel_sequence/__init__.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:54,218 copying colossalai/nn/layer/parallel_sequence/_utils.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:54,223 creating build/lib/colossalai/nn/layer/vanilla 2023-03-10T02:00:54,225 copying colossalai/nn/layer/vanilla/layers.py -> build/lib/colossalai/nn/layer/vanilla 2023-03-10T02:00:54,230 copying colossalai/nn/layer/vanilla/__init__.py -> build/lib/colossalai/nn/layer/vanilla 2023-03-10T02:00:54,235 creating build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:54,236 copying colossalai/nn/layer/parallel_2d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:54,243 copying colossalai/nn/layer/parallel_2d/layers.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:54,250 copying colossalai/nn/layer/parallel_2d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:54,254 copying colossalai/nn/layer/parallel_2d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:54,259 creating build/lib/colossalai/nn/layer/wrapper 2023-03-10T02:00:54,260 copying colossalai/nn/layer/wrapper/__init__.py -> build/lib/colossalai/nn/layer/wrapper 2023-03-10T02:00:54,264 copying colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/lib/colossalai/nn/layer/wrapper 2023-03-10T02:00:54,269 creating build/lib/colossalai/nn/layer/moe 2023-03-10T02:00:54,271 copying colossalai/nn/layer/moe/_operation.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T02:00:54,276 copying colossalai/nn/layer/moe/routers.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T02:00:54,280 copying colossalai/nn/layer/moe/layers.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T02:00:54,285 copying colossalai/nn/layer/moe/utils.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T02:00:54,288 copying colossalai/nn/layer/moe/__init__.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T02:00:54,292 copying colossalai/nn/layer/moe/experts.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T02:00:54,297 creating build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:54,299 copying colossalai/nn/layer/parallel_2p5d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:54,305 copying colossalai/nn/layer/parallel_2p5d/layers.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:54,311 copying colossalai/nn/layer/parallel_2p5d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:54,315 copying colossalai/nn/layer/parallel_2p5d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:54,320 creating build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:54,322 copying colossalai/nn/layer/colossalai_layer/dropout.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:54,325 copying colossalai/nn/layer/colossalai_layer/linear.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:54,330 copying colossalai/nn/layer/colossalai_layer/embedding.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:54,334 copying colossalai/nn/layer/colossalai_layer/normalization.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:54,337 copying colossalai/nn/layer/colossalai_layer/__init__.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:54,341 copying colossalai/nn/layer/colossalai_layer/_utils.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:54,345 creating build/lib/colossalai/cli/benchmark 2023-03-10T02:00:54,347 copying colossalai/cli/benchmark/utils.py -> build/lib/colossalai/cli/benchmark 2023-03-10T02:00:54,352 copying colossalai/cli/benchmark/__init__.py -> build/lib/colossalai/cli/benchmark 2023-03-10T02:00:54,355 copying colossalai/cli/benchmark/benchmark.py -> build/lib/colossalai/cli/benchmark 2023-03-10T02:00:54,359 copying colossalai/cli/benchmark/models.py -> build/lib/colossalai/cli/benchmark 2023-03-10T02:00:54,363 creating build/lib/colossalai/cli/check 2023-03-10T02:00:54,365 copying colossalai/cli/check/__init__.py -> build/lib/colossalai/cli/check 2023-03-10T02:00:54,368 copying colossalai/cli/check/check_installation.py -> build/lib/colossalai/cli/check 2023-03-10T02:00:54,374 creating build/lib/colossalai/cli/launcher 2023-03-10T02:00:54,375 copying colossalai/cli/launcher/run.py -> build/lib/colossalai/cli/launcher 2023-03-10T02:00:54,380 copying colossalai/cli/launcher/hostinfo.py -> build/lib/colossalai/cli/launcher 2023-03-10T02:00:54,383 copying colossalai/cli/launcher/__init__.py -> build/lib/colossalai/cli/launcher 2023-03-10T02:00:54,386 copying colossalai/cli/launcher/multinode_runner.py -> build/lib/colossalai/cli/launcher 2023-03-10T02:00:54,415 creating build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,416 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,421 copying colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,425 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,430 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,435 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,440 copying colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,445 copying colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,449 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,453 copying colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,458 copying colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,462 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,468 copying colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,473 copying colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,477 copying colossalai/kernel/cuda_native/csrc/compat.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,481 copying colossalai/kernel/cuda_native/csrc/type_shim.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,486 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,492 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,496 copying colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,500 copying colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,506 copying colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,511 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,515 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,520 copying colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:54,525 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels 2023-03-10T02:00:54,527 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,530 copying colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,534 copying colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,539 copying colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,544 copying colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,548 copying colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,554 copying colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,558 copying colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,563 copying colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,568 copying colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,574 copying colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,579 copying colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,583 copying colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:54,795 /usr/local/lib/python3.7/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. 2023-03-10T02:00:54,796 setuptools.SetuptoolsDeprecationWarning, 2023-03-10T02:00:54,849 installing to build/bdist.linux-armv7l/wheel 2023-03-10T02:00:54,850 running install 2023-03-10T02:00:54,911 running install_lib 2023-03-10T02:00:54,967 creating build/bdist.linux-armv7l 2023-03-10T02:00:54,969 creating build/bdist.linux-armv7l/wheel 2023-03-10T02:00:54,974 creating build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T02:00:54,977 creating build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T02:00:54,980 copying build/lib/colossalai/testing/pytest_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T02:00:54,984 copying build/lib/colossalai/testing/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T02:00:54,988 copying build/lib/colossalai/testing/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T02:00:54,991 copying build/lib/colossalai/testing/random.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T02:00:54,995 copying build/lib/colossalai/testing/comparison.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T02:00:55,000 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-03-10T02:00:55,002 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:55,004 copying build/lib/colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:55,008 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:55,012 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:55,017 copying build/lib/colossalai/auto_parallel/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:55,021 copying build/lib/colossalai/auto_parallel/checkpoint/operation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:55,026 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T02:00:55,032 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T02:00:55,035 copying build/lib/colossalai/auto_parallel/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T02:00:55,040 copying build/lib/colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T02:00:55,045 copying build/lib/colossalai/auto_parallel/passes/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T02:00:55,049 copying build/lib/colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T02:00:55,055 copying build/lib/colossalai/auto_parallel/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T02:00:55,059 copying build/lib/colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T02:00:55,065 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-03-10T02:00:55,067 copying build/lib/colossalai/auto_parallel/pipeline_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-03-10T02:00:55,071 copying build/lib/colossalai/auto_parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-03-10T02:00:55,076 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:55,078 copying build/lib/colossalai/auto_parallel/tensor_shard/options.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:55,083 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:55,086 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:55,091 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:55,096 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:55,102 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:55,106 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:55,110 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T02:00:55,116 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:55,118 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:55,124 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:55,130 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:55,135 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:55,139 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T02:00:55,144 copying build/lib/colossalai/auto_parallel/tensor_shard/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:55,150 copying build/lib/colossalai/auto_parallel/tensor_shard/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:55,154 copying build/lib/colossalai/auto_parallel/tensor_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:55,158 copying build/lib/colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T02:00:55,165 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,167 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,172 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,177 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,182 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,188 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,192 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,197 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,202 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,206 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,210 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,214 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,217 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,221 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,224 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,230 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,232 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,236 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,240 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,244 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,248 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,252 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,257 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,261 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,266 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,270 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,275 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,280 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,284 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,289 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,293 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,298 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,302 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,306 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,313 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T02:00:55,317 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,322 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,326 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,330 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,333 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,337 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,341 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,344 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,349 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,353 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,358 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,361 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,365 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T02:00:55,370 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:55,372 copying build/lib/colossalai/auto_parallel/meta_profiler/metainfo.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:55,376 copying build/lib/colossalai/auto_parallel/meta_profiler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:55,381 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,382 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,387 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/where.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,391 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,394 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,398 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,404 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,408 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,412 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,416 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,420 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,424 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T02:00:55,427 copying build/lib/colossalai/auto_parallel/meta_profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:55,431 copying build/lib/colossalai/auto_parallel/meta_profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T02:00:55,436 creating build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,438 creating build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T02:00:55,440 copying build/lib/colossalai/utils/model/lazy_init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T02:00:55,445 copying build/lib/colossalai/utils/model/colo_init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T02:00:55,449 copying build/lib/colossalai/utils/model/experimental.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T02:00:55,454 copying build/lib/colossalai/utils/model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T02:00:55,458 copying build/lib/colossalai/utils/model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T02:00:55,463 creating build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-03-10T02:00:55,465 copying build/lib/colossalai/utils/rank_recorder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-03-10T02:00:55,470 copying build/lib/colossalai/utils/rank_recorder/rank_recorder.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-03-10T02:00:55,475 creating build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-03-10T02:00:55,478 copying build/lib/colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-03-10T02:00:55,482 copying build/lib/colossalai/utils/multi_tensor_apply/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-03-10T02:00:55,486 copying build/lib/colossalai/utils/timer.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,491 copying build/lib/colossalai/utils/cuda.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,495 copying build/lib/colossalai/utils/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,501 creating build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T02:00:55,503 copying build/lib/colossalai/utils/checkpoint/module_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T02:00:55,509 copying build/lib/colossalai/utils/checkpoint/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T02:00:55,513 copying build/lib/colossalai/utils/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T02:00:55,517 copying build/lib/colossalai/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,524 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T02:00:55,526 copying build/lib/colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T02:00:55,532 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T02:00:55,534 copying build/lib/colossalai/utils/profiler/legacy/pcie_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T02:00:55,539 copying build/lib/colossalai/utils/profiler/legacy/prof_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T02:00:55,544 copying build/lib/colossalai/utils/profiler/legacy/comm_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T02:00:55,549 copying build/lib/colossalai/utils/profiler/legacy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T02:00:55,553 copying build/lib/colossalai/utils/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T02:00:55,558 copying build/lib/colossalai/utils/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T02:00:55,562 copying build/lib/colossalai/utils/profiler/extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T02:00:55,568 creating build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,570 copying build/lib/colossalai/utils/checkpoint_io/convertor.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,575 copying build/lib/colossalai/utils/checkpoint_io/reader.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,580 copying build/lib/colossalai/utils/checkpoint_io/io.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,585 copying build/lib/colossalai/utils/checkpoint_io/meta.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,589 copying build/lib/colossalai/utils/checkpoint_io/distributed.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,594 copying build/lib/colossalai/utils/checkpoint_io/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,599 copying build/lib/colossalai/utils/checkpoint_io/constant.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,603 copying build/lib/colossalai/utils/checkpoint_io/backend.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,607 copying build/lib/colossalai/utils/checkpoint_io/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,611 copying build/lib/colossalai/utils/checkpoint_io/writer.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T02:00:55,616 copying build/lib/colossalai/utils/memory.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,622 creating build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-03-10T02:00:55,624 copying build/lib/colossalai/utils/tensor_detector/tensor_detector.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-03-10T02:00:55,630 copying build/lib/colossalai/utils/tensor_detector/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-03-10T02:00:55,635 creating build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T02:00:55,637 copying build/lib/colossalai/utils/data_sampler/base_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T02:00:55,642 copying build/lib/colossalai/utils/data_sampler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T02:00:55,646 copying build/lib/colossalai/utils/data_sampler/data_parallel_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T02:00:55,650 copying build/lib/colossalai/utils/activation_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,655 copying build/lib/colossalai/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,660 copying build/lib/colossalai/utils/checkpointing.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T02:00:55,666 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T02:00:55,668 copying build/lib/colossalai/pipeline/layer_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T02:00:55,674 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T02:00:55,676 copying build/lib/colossalai/pipeline/rpc/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T02:00:55,681 copying build/lib/colossalai/pipeline/rpc/_pipeline_base.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T02:00:55,689 copying build/lib/colossalai/pipeline/rpc/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T02:00:55,694 copying build/lib/colossalai/pipeline/rpc/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T02:00:55,699 copying build/lib/colossalai/pipeline/pipeline_process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T02:00:55,703 copying build/lib/colossalai/pipeline/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T02:00:55,709 copying build/lib/colossalai/pipeline/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T02:00:55,713 copying build/lib/colossalai/pipeline/pipelinable.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T02:00:55,720 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-03-10T02:00:55,723 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-03-10T02:00:55,726 copying build/lib/colossalai/pipeline/middleware/adaptor/fx.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-03-10T02:00:55,731 copying build/lib/colossalai/pipeline/middleware/adaptor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-03-10T02:00:55,736 copying build/lib/colossalai/pipeline/middleware/topo.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-03-10T02:00:55,741 copying build/lib/colossalai/pipeline/middleware/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-03-10T02:00:55,746 creating build/bdist.linux-armv7l/wheel/colossalai/zero 2023-03-10T02:00:55,749 creating build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T02:00:55,752 copying build/lib/colossalai/zero/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T02:00:55,756 copying build/lib/colossalai/zero/utils/gemini_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T02:00:55,760 copying build/lib/colossalai/zero/utils/zero_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T02:00:55,766 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T02:00:55,769 copying build/lib/colossalai/zero/sharded_param/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T02:00:55,773 copying build/lib/colossalai/zero/sharded_param/sharded_param.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T02:00:55,777 copying build/lib/colossalai/zero/sharded_param/sharded_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T02:00:55,783 creating build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-03-10T02:00:55,784 copying build/lib/colossalai/zero/init_ctx/init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-03-10T02:00:55,789 copying build/lib/colossalai/zero/init_ctx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-03-10T02:00:55,793 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T02:00:55,795 copying build/lib/colossalai/zero/sharded_optim/sharded_optim_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T02:00:55,802 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:55,803 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/bucket_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:55,807 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:55,811 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/parameter_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:55,815 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/base_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:55,818 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:55,822 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/gradient_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T02:00:55,825 copying build/lib/colossalai/zero/sharded_optim/low_level_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T02:00:55,831 copying build/lib/colossalai/zero/sharded_optim/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T02:00:55,834 copying build/lib/colossalai/zero/sharded_optim/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T02:00:55,840 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T02:00:55,841 copying build/lib/colossalai/zero/sharded_model/reduce_scatter.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T02:00:55,846 copying build/lib/colossalai/zero/sharded_model/sharded_model_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T02:00:55,852 copying build/lib/colossalai/zero/sharded_model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T02:00:55,856 copying build/lib/colossalai/zero/sharded_model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T02:00:55,859 copying build/lib/colossalai/zero/sharded_model/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T02:00:55,863 copying build/lib/colossalai/zero/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero 2023-03-10T02:00:55,867 creating build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T02:00:55,869 copying build/lib/colossalai/zero/shard_utils/commons.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T02:00:55,873 copying build/lib/colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T02:00:55,877 copying build/lib/colossalai/zero/shard_utils/tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T02:00:55,880 copying build/lib/colossalai/zero/shard_utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T02:00:55,884 copying build/lib/colossalai/zero/shard_utils/base_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T02:00:55,887 copying build/lib/colossalai/global_variables.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T02:00:55,891 copying build/lib/colossalai/version.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T02:00:55,896 creating build/bdist.linux-armv7l/wheel/colossalai/builder 2023-03-10T02:00:55,897 copying build/lib/colossalai/builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/builder 2023-03-10T02:00:55,901 copying build/lib/colossalai/builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/builder 2023-03-10T02:00:55,906 creating build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T02:00:55,909 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:55,911 copying build/lib/colossalai/fx/tracer/_tracer_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:55,915 copying build/lib/colossalai/fx/tracer/_meta_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:55,920 copying build/lib/colossalai/fx/tracer/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:55,924 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-03-10T02:00:55,927 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:55,929 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:55,933 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:55,938 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:55,941 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:55,945 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T02:00:55,951 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:55,952 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:55,956 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:55,960 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:55,964 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T02:00:55,967 copying build/lib/colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-03-10T02:00:55,971 copying build/lib/colossalai/fx/tracer/_symbolic_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:55,975 copying build/lib/colossalai/fx/tracer/experimental.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:55,982 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-03-10T02:00:55,985 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:55,987 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:55,992 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:55,995 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:55,999 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:56,003 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:56,007 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:56,011 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:56,014 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T02:00:56,020 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,022 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,026 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,030 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,034 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,037 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,042 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,046 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,049 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T02:00:56,053 copying build/lib/colossalai/fx/tracer/meta_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-03-10T02:00:56,057 copying build/lib/colossalai/fx/tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:56,060 copying build/lib/colossalai/fx/tracer/tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T02:00:56,067 creating build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-03-10T02:00:56,068 copying build/lib/colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-03-10T02:00:56,075 copying build/lib/colossalai/fx/codegen/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-03-10T02:00:56,079 copying build/lib/colossalai/fx/proxy.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T02:00:56,084 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,087 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T02:00:56,090 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,092 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,096 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,099 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,103 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,106 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,111 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,114 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,118 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,123 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,126 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,130 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T02:00:56,134 copying build/lib/colossalai/fx/profiler/experimental/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T02:00:56,137 copying build/lib/colossalai/fx/profiler/experimental/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T02:00:56,141 copying build/lib/colossalai/fx/profiler/experimental/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T02:00:56,144 copying build/lib/colossalai/fx/profiler/experimental/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T02:00:56,150 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,152 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,156 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,159 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,163 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,167 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,170 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,174 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,177 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,181 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T02:00:56,184 copying build/lib/colossalai/fx/profiler/experimental/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T02:00:56,187 copying build/lib/colossalai/fx/profiler/tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,192 copying build/lib/colossalai/fx/profiler/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,196 copying build/lib/colossalai/fx/profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,200 copying build/lib/colossalai/fx/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,204 copying build/lib/colossalai/fx/profiler/memory_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,208 copying build/lib/colossalai/fx/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,211 copying build/lib/colossalai/fx/profiler/opcount.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,216 copying build/lib/colossalai/fx/profiler/dataflow.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T02:00:56,221 creating build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,223 copying build/lib/colossalai/fx/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,228 copying build/lib/colossalai/fx/passes/passes_for_gpt2_test.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,233 copying build/lib/colossalai/fx/passes/concrete_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,237 copying build/lib/colossalai/fx/passes/shard_1d_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,242 copying build/lib/colossalai/fx/passes/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,246 copying build/lib/colossalai/fx/passes/split_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,250 copying build/lib/colossalai/fx/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,254 copying build/lib/colossalai/fx/passes/adding_split_node_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T02:00:56,259 copying build/lib/colossalai/fx/_compatibility.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T02:00:56,262 copying build/lib/colossalai/fx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T02:00:56,265 copying build/lib/colossalai/fx/_meta_registrations.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T02:00:56,270 copying build/lib/colossalai/fx/graph_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T02:00:56,275 creating build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T02:00:56,277 copying build/lib/colossalai/context/singleton_meta.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T02:00:56,281 copying build/lib/colossalai/context/config.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T02:00:56,284 copying build/lib/colossalai/context/parallel_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T02:00:56,289 copying build/lib/colossalai/context/moe_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T02:00:56,294 creating build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,296 copying build/lib/colossalai/context/process_group_initializer/initializer_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,300 copying build/lib/colossalai/context/process_group_initializer/initializer_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,304 copying build/lib/colossalai/context/process_group_initializer/initializer_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,309 copying build/lib/colossalai/context/process_group_initializer/initializer_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,313 copying build/lib/colossalai/context/process_group_initializer/initializer_sequence.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,316 copying build/lib/colossalai/context/process_group_initializer/initializer_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,320 copying build/lib/colossalai/context/process_group_initializer/initializer_pipeline.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,323 copying build/lib/colossalai/context/process_group_initializer/initializer_model.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,327 copying build/lib/colossalai/context/process_group_initializer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,330 copying build/lib/colossalai/context/process_group_initializer/initializer_data.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,334 copying build/lib/colossalai/context/process_group_initializer/process_group_initializer.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T02:00:56,337 copying build/lib/colossalai/context/parallel_mode.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T02:00:56,341 copying build/lib/colossalai/context/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T02:00:56,345 creating build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T02:00:56,347 copying build/lib/colossalai/context/random/_helper.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T02:00:56,351 copying build/lib/colossalai/context/random/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T02:00:56,354 copying build/lib/colossalai/context/random/seed_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T02:00:56,358 copying build/lib/colossalai/core.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T02:00:56,361 copying build/lib/colossalai/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T02:00:56,366 copying build/lib/colossalai/constants.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T02:00:56,371 creating build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,373 copying build/lib/colossalai/tensor/comm_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,378 copying build/lib/colossalai/tensor/dist_spec_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,383 copying build/lib/colossalai/tensor/const.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,386 copying build/lib/colossalai/tensor/compute_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,390 copying build/lib/colossalai/tensor/shape_consistency.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,395 copying build/lib/colossalai/tensor/sharding_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,400 copying build/lib/colossalai/tensor/process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,404 copying build/lib/colossalai/tensor/distspec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,408 copying build/lib/colossalai/tensor/op_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,411 copying build/lib/colossalai/tensor/colo_parameter.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,416 creating build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T02:00:56,418 copying build/lib/colossalai/tensor/d_tensor/comm_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T02:00:56,423 copying build/lib/colossalai/tensor/d_tensor/sharding_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T02:00:56,427 copying build/lib/colossalai/tensor/d_tensor/d_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T02:00:56,431 copying build/lib/colossalai/tensor/d_tensor/misc.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T02:00:56,435 copying build/lib/colossalai/tensor/d_tensor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T02:00:56,437 copying build/lib/colossalai/tensor/d_tensor/layout.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T02:00:56,441 copying build/lib/colossalai/tensor/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,446 copying build/lib/colossalai/tensor/colo_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,450 copying build/lib/colossalai/tensor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,453 copying build/lib/colossalai/tensor/param_op_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,457 copying build/lib/colossalai/tensor/tensor_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T02:00:56,462 creating build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-03-10T02:00:56,465 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,467 copying build/lib/colossalai/kernel/op_builder/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,470 copying build/lib/colossalai/kernel/op_builder/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,474 copying build/lib/colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,477 copying build/lib/colossalai/kernel/op_builder/multi_head_attn.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,481 copying build/lib/colossalai/kernel/op_builder/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,484 copying build/lib/colossalai/kernel/op_builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,488 copying build/lib/colossalai/kernel/op_builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,492 copying build/lib/colossalai/kernel/op_builder/fused_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,495 copying build/lib/colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,499 copying build/lib/colossalai/kernel/op_builder/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T02:00:56,503 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T02:00:56,505 copying build/lib/colossalai/kernel/cuda_native/multihead_attention.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T02:00:56,509 copying build/lib/colossalai/kernel/cuda_native/flash_attention.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T02:00:56,516 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,517 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,521 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,525 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,530 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,534 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,539 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,543 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,547 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,551 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels 2023-03-10T02:00:56,554 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,556 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,560 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,563 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,567 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,570 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,574 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,578 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,582 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,585 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,589 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,593 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,596 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T02:00:56,600 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,604 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,609 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,615 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,620 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,626 copying build/lib/colossalai/kernel/cuda_native/csrc/compat.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,630 copying build/lib/colossalai/kernel/cuda_native/csrc/type_shim.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,635 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,641 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,645 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,649 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,654 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,660 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,664 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,669 copying build/lib/colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T02:00:56,673 copying build/lib/colossalai/kernel/cuda_native/layer_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T02:00:56,677 copying build/lib/colossalai/kernel/cuda_native/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T02:00:56,681 copying build/lib/colossalai/kernel/cuda_native/scaled_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T02:00:56,686 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T02:00:56,689 copying build/lib/colossalai/kernel/jit/option.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T02:00:56,693 copying build/lib/colossalai/kernel/jit/bias_dropout_add.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T02:00:56,697 copying build/lib/colossalai/kernel/jit/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T02:00:56,701 copying build/lib/colossalai/kernel/jit/bias_gelu.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T02:00:56,705 copying build/lib/colossalai/kernel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-03-10T02:00:56,710 creating build/bdist.linux-armv7l/wheel/colossalai/amp 2023-03-10T02:00:56,713 creating build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T02:00:56,716 copying build/lib/colossalai/amp/torch_amp/torch_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T02:00:56,720 copying build/lib/colossalai/amp/torch_amp/_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T02:00:56,725 copying build/lib/colossalai/amp/torch_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T02:00:56,730 copying build/lib/colossalai/amp/amp_type.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-03-10T02:00:56,734 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T02:00:56,737 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:56,739 copying build/lib/colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:56,742 copying build/lib/colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:56,746 copying build/lib/colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:56,749 copying build/lib/colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T02:00:56,753 copying build/lib/colossalai/amp/naive_amp/_fp16_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T02:00:56,757 copying build/lib/colossalai/amp/naive_amp/naive_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T02:00:56,761 copying build/lib/colossalai/amp/naive_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T02:00:56,765 copying build/lib/colossalai/amp/naive_amp/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T02:00:56,768 copying build/lib/colossalai/amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-03-10T02:00:56,773 creating build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-03-10T02:00:56,774 copying build/lib/colossalai/amp/apex_amp/apex_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-03-10T02:00:56,778 copying build/lib/colossalai/amp/apex_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-03-10T02:00:56,782 creating build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T02:00:56,784 copying build/lib/colossalai/device/device_mesh.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T02:00:56,789 copying build/lib/colossalai/device/calc_pipeline_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T02:00:56,793 copying build/lib/colossalai/device/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T02:00:56,796 copying build/lib/colossalai/device/alpha_beta_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T02:00:56,802 creating build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,805 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T02:00:56,807 copying build/lib/colossalai/gemini/ophooks/_shard_param_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T02:00:56,810 copying build/lib/colossalai/gemini/ophooks/_shard_grad_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T02:00:56,814 copying build/lib/colossalai/gemini/ophooks/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T02:00:56,817 copying build/lib/colossalai/gemini/ophooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T02:00:56,821 copying build/lib/colossalai/gemini/ophooks/runtime_mem_tracer_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T02:00:56,826 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T02:00:56,827 copying build/lib/colossalai/gemini/chunk/search_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T02:00:56,832 copying build/lib/colossalai/gemini/chunk/chunk.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T02:00:56,837 copying build/lib/colossalai/gemini/chunk/manager.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T02:00:56,841 copying build/lib/colossalai/gemini/chunk/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T02:00:56,845 copying build/lib/colossalai/gemini/chunk/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T02:00:56,848 copying build/lib/colossalai/gemini/stateful_tensor_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,852 copying build/lib/colossalai/gemini/tensor_placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,856 copying build/lib/colossalai/gemini/gemini_context.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,859 copying build/lib/colossalai/gemini/gemini_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,863 copying build/lib/colossalai/gemini/placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,869 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,870 copying build/lib/colossalai/gemini/memory_tracer/memory_monitor.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,874 copying build/lib/colossalai/gemini/memory_tracer/memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,877 copying build/lib/colossalai/gemini/memory_tracer/runtime_mem_tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,881 copying build/lib/colossalai/gemini/memory_tracer/memory_stats.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,885 copying build/lib/colossalai/gemini/memory_tracer/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,888 copying build/lib/colossalai/gemini/memory_tracer/param_runtime_order.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,892 copying build/lib/colossalai/gemini/memory_tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,895 copying build/lib/colossalai/gemini/memory_tracer/chunk_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,899 copying build/lib/colossalai/gemini/memory_tracer/static_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T02:00:56,902 copying build/lib/colossalai/gemini/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,906 copying build/lib/colossalai/gemini/tensor_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,910 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-03-10T02:00:56,912 copying build/lib/colossalai/gemini/paramhooks/_param_hookmgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-03-10T02:00:56,916 copying build/lib/colossalai/gemini/paramhooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-03-10T02:00:56,919 copying build/lib/colossalai/gemini/stateful_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T02:00:56,923 copying build/lib/colossalai/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T02:00:56,927 creating build/bdist.linux-armv7l/wheel/colossalai/logging 2023-03-10T02:00:56,929 copying build/lib/colossalai/logging/logger.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-03-10T02:00:56,933 copying build/lib/colossalai/logging/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-03-10T02:00:56,938 creating build/bdist.linux-armv7l/wheel/colossalai/_C 2023-03-10T02:00:56,939 copying build/lib/colossalai/_C/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_C 2023-03-10T02:00:56,944 creating build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T02:00:56,945 copying build/lib/colossalai/booster/accelerator.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T02:00:56,949 copying build/lib/colossalai/booster/plugin.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T02:00:56,952 copying build/lib/colossalai/booster/booster.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T02:00:56,955 copying build/lib/colossalai/booster/precision.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T02:00:56,959 copying build/lib/colossalai/booster/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T02:00:56,962 copying build/lib/colossalai/booster/environment_table.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T02:00:56,967 creating build/bdist.linux-armv7l/wheel/colossalai/engine 2023-03-10T02:00:56,970 creating build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,972 copying build/lib/colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,975 copying build/lib/colossalai/engine/gradient_handler/_base_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,979 copying build/lib/colossalai/engine/gradient_handler/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,982 copying build/lib/colossalai/engine/gradient_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,986 copying build/lib/colossalai/engine/gradient_handler/_zero_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,989 copying build/lib/colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,993 copying build/lib/colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:56,996 copying build/lib/colossalai/engine/gradient_handler/_moe_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T02:00:57,000 copying build/lib/colossalai/engine/_base_engine.py -> build/bdist.linux-armv7l/wheel/colossalai/engine 2023-03-10T02:00:57,006 creating build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T02:00:57,007 copying build/lib/colossalai/engine/schedule/_base_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T02:00:57,012 copying build/lib/colossalai/engine/schedule/_non_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T02:00:57,016 copying build/lib/colossalai/engine/schedule/_pipeline_schedule_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T02:00:57,020 copying build/lib/colossalai/engine/schedule/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T02:00:57,026 copying build/lib/colossalai/engine/schedule/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T02:00:57,030 copying build/lib/colossalai/engine/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine 2023-03-10T02:00:57,035 creating build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-03-10T02:00:57,036 copying build/lib/colossalai/engine/gradient_accumulation/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-03-10T02:00:57,040 copying build/lib/colossalai/engine/gradient_accumulation/_gradient_accumulation.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-03-10T02:00:57,046 creating build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-03-10T02:00:57,049 creating build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,051 copying build/lib/colossalai/trainer/hooks/_base_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,054 copying build/lib/colossalai/trainer/hooks/_lr_scheduler_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,058 copying build/lib/colossalai/trainer/hooks/_checkpoint_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,062 copying build/lib/colossalai/trainer/hooks/_commons_.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,065 copying build/lib/colossalai/trainer/hooks/_log_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,070 copying build/lib/colossalai/trainer/hooks/_metric_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,075 copying build/lib/colossalai/trainer/hooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T02:00:57,078 copying build/lib/colossalai/trainer/_trainer.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-03-10T02:00:57,083 copying build/lib/colossalai/trainer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-03-10T02:00:57,088 creating build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T02:00:57,089 copying build/lib/colossalai/communication/collective.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T02:00:57,094 copying build/lib/colossalai/communication/ring.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T02:00:57,098 copying build/lib/colossalai/communication/p2p.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T02:00:57,103 copying build/lib/colossalai/communication/p2p_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T02:00:57,107 copying build/lib/colossalai/communication/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T02:00:57,111 copying build/lib/colossalai/communication/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T02:00:57,115 creating build/bdist.linux-armv7l/wheel/colossalai/registry 2023-03-10T02:00:57,117 copying build/lib/colossalai/registry/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/registry 2023-03-10T02:00:57,122 copying build/lib/colossalai/registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/registry 2023-03-10T02:00:57,126 creating build/bdist.linux-armv7l/wheel/colossalai/nn 2023-03-10T02:00:57,129 creating build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,131 copying build/lib/colossalai/nn/optimizer/gemini_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,134 copying build/lib/colossalai/nn/optimizer/lars.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,138 copying build/lib/colossalai/nn/optimizer/hybrid_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,142 copying build/lib/colossalai/nn/optimizer/fused_sgd.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,147 copying build/lib/colossalai/nn/optimizer/zero_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,152 copying build/lib/colossalai/nn/optimizer/colossalai_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,155 copying build/lib/colossalai/nn/optimizer/fused_lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,160 copying build/lib/colossalai/nn/optimizer/nvme_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,164 copying build/lib/colossalai/nn/optimizer/lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,168 copying build/lib/colossalai/nn/optimizer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,172 copying build/lib/colossalai/nn/optimizer/fused_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,176 copying build/lib/colossalai/nn/optimizer/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T02:00:57,182 creating build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T02:00:57,184 copying build/lib/colossalai/nn/metric/accuracy_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T02:00:57,188 copying build/lib/colossalai/nn/metric/accuracy_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T02:00:57,193 copying build/lib/colossalai/nn/metric/accuracy_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T02:00:57,197 copying build/lib/colossalai/nn/metric/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T02:00:57,200 copying build/lib/colossalai/nn/metric/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T02:00:57,205 creating build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T02:00:57,206 copying build/lib/colossalai/nn/loss/loss_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T02:00:57,211 copying build/lib/colossalai/nn/loss/loss_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T02:00:57,215 copying build/lib/colossalai/nn/loss/loss_moe.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T02:00:57,218 copying build/lib/colossalai/nn/loss/loss_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T02:00:57,222 copying build/lib/colossalai/nn/loss/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T02:00:57,226 copying build/lib/colossalai/nn/loss/loss_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T02:00:57,232 creating build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,233 copying build/lib/colossalai/nn/_ops/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,237 copying build/lib/colossalai/nn/_ops/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,241 copying build/lib/colossalai/nn/_ops/batch_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,244 copying build/lib/colossalai/nn/_ops/loss.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,248 copying build/lib/colossalai/nn/_ops/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,252 copying build/lib/colossalai/nn/_ops/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,256 copying build/lib/colossalai/nn/_ops/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,260 copying build/lib/colossalai/nn/_ops/element_wise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,264 copying build/lib/colossalai/nn/_ops/view.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,268 copying build/lib/colossalai/nn/_ops/embedding_bag.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,272 copying build/lib/colossalai/nn/_ops/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T02:00:57,276 copying build/lib/colossalai/nn/init.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-03-10T02:00:57,282 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T02:00:57,283 copying build/lib/colossalai/nn/parallel/zero_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T02:00:57,287 copying build/lib/colossalai/nn/parallel/gemini_parallel.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T02:00:57,291 copying build/lib/colossalai/nn/parallel/reducer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T02:00:57,296 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T02:00:57,297 copying build/lib/colossalai/nn/parallel/layers/module_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T02:00:57,302 copying build/lib/colossalai/nn/parallel/layers/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T02:00:57,305 copying build/lib/colossalai/nn/parallel/layers/colo_module.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T02:00:57,309 copying build/lib/colossalai/nn/parallel/layers/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T02:00:57,313 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,315 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,319 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,323 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,326 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,331 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,335 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,340 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,344 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,348 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T02:00:57,352 copying build/lib/colossalai/nn/parallel/layers/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T02:00:57,356 copying build/lib/colossalai/nn/parallel/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T02:00:57,360 copying build/lib/colossalai/nn/parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T02:00:57,364 copying build/lib/colossalai/nn/parallel/data_parallel.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T02:00:57,370 copying build/lib/colossalai/nn/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-03-10T02:00:57,374 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-03-10T02:00:57,377 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:57,379 copying build/lib/colossalai/nn/layer/parallel_3d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:57,385 copying build/lib/colossalai/nn/layer/parallel_3d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:57,393 copying build/lib/colossalai/nn/layer/parallel_3d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:57,396 copying build/lib/colossalai/nn/layer/parallel_3d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T02:00:57,401 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-03-10T02:00:57,402 copying build/lib/colossalai/nn/layer/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-03-10T02:00:57,406 copying build/lib/colossalai/nn/layer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-03-10T02:00:57,410 copying build/lib/colossalai/nn/layer/base_layer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-03-10T02:00:57,414 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:57,415 copying build/lib/colossalai/nn/layer/parallel_1d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:57,419 copying build/lib/colossalai/nn/layer/parallel_1d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:57,425 copying build/lib/colossalai/nn/layer/parallel_1d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:57,429 copying build/lib/colossalai/nn/layer/parallel_1d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T02:00:57,434 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:57,436 copying build/lib/colossalai/nn/layer/parallel_sequence/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:57,440 copying build/lib/colossalai/nn/layer/parallel_sequence/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:57,445 copying build/lib/colossalai/nn/layer/parallel_sequence/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:57,450 copying build/lib/colossalai/nn/layer/parallel_sequence/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T02:00:57,455 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-03-10T02:00:57,457 copying build/lib/colossalai/nn/layer/vanilla/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-03-10T02:00:57,463 copying build/lib/colossalai/nn/layer/vanilla/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-03-10T02:00:57,467 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:57,469 copying build/lib/colossalai/nn/layer/parallel_2d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:57,476 copying build/lib/colossalai/nn/layer/parallel_2d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:57,482 copying build/lib/colossalai/nn/layer/parallel_2d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:57,486 copying build/lib/colossalai/nn/layer/parallel_2d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T02:00:57,491 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-03-10T02:00:57,492 copying build/lib/colossalai/nn/layer/wrapper/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-03-10T02:00:57,496 copying build/lib/colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-03-10T02:00:57,500 copying build/lib/colossalai/nn/layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-03-10T02:00:57,505 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T02:00:57,506 copying build/lib/colossalai/nn/layer/moe/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T02:00:57,511 copying build/lib/colossalai/nn/layer/moe/routers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T02:00:57,515 copying build/lib/colossalai/nn/layer/moe/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T02:00:57,520 copying build/lib/colossalai/nn/layer/moe/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T02:00:57,523 copying build/lib/colossalai/nn/layer/moe/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T02:00:57,527 copying build/lib/colossalai/nn/layer/moe/experts.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T02:00:57,532 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:57,534 copying build/lib/colossalai/nn/layer/parallel_2p5d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:57,540 copying build/lib/colossalai/nn/layer/parallel_2p5d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:57,547 copying build/lib/colossalai/nn/layer/parallel_2p5d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:57,551 copying build/lib/colossalai/nn/layer/parallel_2p5d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T02:00:57,555 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:57,557 copying build/lib/colossalai/nn/layer/colossalai_layer/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:57,561 copying build/lib/colossalai/nn/layer/colossalai_layer/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:57,565 copying build/lib/colossalai/nn/layer/colossalai_layer/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:57,570 copying build/lib/colossalai/nn/layer/colossalai_layer/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:57,573 copying build/lib/colossalai/nn/layer/colossalai_layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:57,577 copying build/lib/colossalai/nn/layer/colossalai_layer/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T02:00:57,582 creating build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,584 copying build/lib/colossalai/nn/lr_scheduler/multistep.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,587 copying build/lib/colossalai/nn/lr_scheduler/poly.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,591 copying build/lib/colossalai/nn/lr_scheduler/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,595 copying build/lib/colossalai/nn/lr_scheduler/torch.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,599 copying build/lib/colossalai/nn/lr_scheduler/cosine.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,603 copying build/lib/colossalai/nn/lr_scheduler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,606 copying build/lib/colossalai/nn/lr_scheduler/onecycle.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,610 copying build/lib/colossalai/nn/lr_scheduler/delayed.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T02:00:57,615 creating build/bdist.linux-armv7l/wheel/colossalai/cli 2023-03-10T02:00:57,619 creating build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T02:00:57,621 copying build/lib/colossalai/cli/benchmark/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T02:00:57,625 copying build/lib/colossalai/cli/benchmark/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T02:00:57,629 copying build/lib/colossalai/cli/benchmark/benchmark.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T02:00:57,633 copying build/lib/colossalai/cli/benchmark/models.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T02:00:57,636 copying build/lib/colossalai/cli/cli.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-03-10T02:00:57,641 creating build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-03-10T02:00:57,643 copying build/lib/colossalai/cli/check/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-03-10T02:00:57,647 copying build/lib/colossalai/cli/check/check_installation.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-03-10T02:00:57,651 copying build/lib/colossalai/cli/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-03-10T02:00:57,655 creating build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T02:00:57,657 copying build/lib/colossalai/cli/launcher/run.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T02:00:57,662 copying build/lib/colossalai/cli/launcher/hostinfo.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T02:00:57,666 copying build/lib/colossalai/cli/launcher/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T02:00:57,669 copying build/lib/colossalai/cli/launcher/multinode_runner.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T02:00:57,675 creating build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,676 copying build/lib/op_builder/layernorm.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,681 copying build/lib/op_builder/moe.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,684 copying build/lib/op_builder/scaled_masked_softmax.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,687 copying build/lib/op_builder/multi_head_attn.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,691 copying build/lib/op_builder/utils.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,695 copying build/lib/op_builder/__init__.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,699 copying build/lib/op_builder/builder.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,703 copying build/lib/op_builder/fused_optim.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,707 copying build/lib/op_builder/scaled_upper_triangle_masked_softmax.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,710 copying build/lib/op_builder/cpu_adam.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T02:00:57,713 running install_egg_info 2023-03-10T02:00:57,851 running egg_info 2023-03-10T02:00:57,902 writing colossalai.egg-info/PKG-INFO 2023-03-10T02:00:57,907 writing dependency_links to colossalai.egg-info/dependency_links.txt 2023-03-10T02:00:57,911 writing entry points to colossalai.egg-info/entry_points.txt 2023-03-10T02:00:57,915 writing requirements to colossalai.egg-info/requires.txt 2023-03-10T02:00:57,918 writing top-level names to colossalai.egg-info/top_level.txt 2023-03-10T02:00:58,178 reading manifest file 'colossalai.egg-info/SOURCES.txt' 2023-03-10T02:00:58,242 reading manifest template 'MANIFEST.in' 2023-03-10T02:01:00,243 warning: no files found matching '*.tr' under directory 'colossalai' 2023-03-10T02:01:01,234 warning: no files found matching '*.cc' under directory 'colossalai' 2023-03-10T02:01:01,762 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-03-10T02:01:01,779 adding license file 'LICENSE' 2023-03-10T02:01:01,875 writing manifest file 'colossalai.egg-info/SOURCES.txt' 2023-03-10T02:01:01,882 Copying colossalai.egg-info to build/bdist.linux-armv7l/wheel/colossalai-0.2.6-py3.7.egg-info 2023-03-10T02:01:01,913 running install_scripts 2023-03-10T02:01:01,966 adding license file "LICENSE" (matched pattern "LICEN[CS]E*") 2023-03-10T02:01:01,979 creating build/bdist.linux-armv7l/wheel/colossalai-0.2.6.dist-info/WHEEL 2023-03-10T02:01:01,985 creating '/tmp/pip-wheel-tu37a4hr/colossalai-0.2.6-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2023-03-10T02:01:01,992 adding 'colossalai/__init__.py' 2023-03-10T02:01:01,995 adding 'colossalai/constants.py' 2023-03-10T02:01:01,997 adding 'colossalai/core.py' 2023-03-10T02:01:02,000 adding 'colossalai/global_variables.py' 2023-03-10T02:01:02,007 adding 'colossalai/initialize.py' 2023-03-10T02:01:02,009 adding 'colossalai/version.py' 2023-03-10T02:01:02,013 adding 'colossalai/_C/__init__.py' 2023-03-10T02:01:02,018 adding 'colossalai/amp/__init__.py' 2023-03-10T02:01:02,020 adding 'colossalai/amp/amp_type.py' 2023-03-10T02:01:02,025 adding 'colossalai/amp/apex_amp/__init__.py' 2023-03-10T02:01:02,030 adding 'colossalai/amp/apex_amp/apex_amp.py' 2023-03-10T02:01:02,034 adding 'colossalai/amp/naive_amp/__init__.py' 2023-03-10T02:01:02,039 adding 'colossalai/amp/naive_amp/_fp16_optimizer.py' 2023-03-10T02:01:02,042 adding 'colossalai/amp/naive_amp/_utils.py' 2023-03-10T02:01:02,046 adding 'colossalai/amp/naive_amp/naive_amp.py' 2023-03-10T02:01:02,050 adding 'colossalai/amp/naive_amp/grad_scaler/__init__.py' 2023-03-10T02:01:02,052 adding 'colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py' 2023-03-10T02:01:02,055 adding 'colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py' 2023-03-10T02:01:02,058 adding 'colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py' 2023-03-10T02:01:02,062 adding 'colossalai/amp/torch_amp/__init__.py' 2023-03-10T02:01:02,070 adding 'colossalai/amp/torch_amp/_grad_scaler.py' 2023-03-10T02:01:02,074 adding 'colossalai/amp/torch_amp/torch_amp.py' 2023-03-10T02:01:02,077 adding 'colossalai/auto_parallel/__init__.py' 2023-03-10T02:01:02,081 adding 'colossalai/auto_parallel/checkpoint/__init__.py' 2023-03-10T02:01:02,083 adding 'colossalai/auto_parallel/checkpoint/build_c_ext.py' 2023-03-10T02:01:02,087 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_base.py' 2023-03-10T02:01:02,091 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py' 2023-03-10T02:01:02,096 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py' 2023-03-10T02:01:02,100 adding 'colossalai/auto_parallel/checkpoint/operation.py' 2023-03-10T02:01:02,103 adding 'colossalai/auto_parallel/meta_profiler/__init__.py' 2023-03-10T02:01:02,106 adding 'colossalai/auto_parallel/meta_profiler/constants.py' 2023-03-10T02:01:02,109 adding 'colossalai/auto_parallel/meta_profiler/metainfo.py' 2023-03-10T02:01:02,111 adding 'colossalai/auto_parallel/meta_profiler/registry.py' 2023-03-10T02:01:02,115 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py' 2023-03-10T02:01:02,118 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/activation.py' 2023-03-10T02:01:02,121 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py' 2023-03-10T02:01:02,125 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/conv.py' 2023-03-10T02:01:02,128 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py' 2023-03-10T02:01:02,134 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/linear.py' 2023-03-10T02:01:02,137 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py' 2023-03-10T02:01:02,140 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/norm.py' 2023-03-10T02:01:02,144 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py' 2023-03-10T02:01:02,147 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py' 2023-03-10T02:01:02,150 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/where.py' 2023-03-10T02:01:02,153 adding 'colossalai/auto_parallel/passes/__init__.py' 2023-03-10T02:01:02,157 adding 'colossalai/auto_parallel/passes/comm_metainfo_pass.py' 2023-03-10T02:01:02,159 adding 'colossalai/auto_parallel/passes/constants.py' 2023-03-10T02:01:02,162 adding 'colossalai/auto_parallel/passes/meta_info_prop.py' 2023-03-10T02:01:02,166 adding 'colossalai/auto_parallel/passes/runtime_apply_pass.py' 2023-03-10T02:01:02,173 adding 'colossalai/auto_parallel/passes/runtime_preparation_pass.py' 2023-03-10T02:01:02,177 adding 'colossalai/auto_parallel/pipeline_shard/__init__.py' 2023-03-10T02:01:02,181 adding 'colossalai/auto_parallel/tensor_shard/__init__.py' 2023-03-10T02:01:02,183 adding 'colossalai/auto_parallel/tensor_shard/constants.py' 2023-03-10T02:01:02,188 adding 'colossalai/auto_parallel/tensor_shard/initialize.py' 2023-03-10T02:01:02,191 adding 'colossalai/auto_parallel/tensor_shard/options.py' 2023-03-10T02:01:02,195 adding 'colossalai/auto_parallel/tensor_shard/sharding_strategy.py' 2023-03-10T02:01:02,200 adding 'colossalai/auto_parallel/tensor_shard/node_handler/__init__.py' 2023-03-10T02:01:02,203 adding 'colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py' 2023-03-10T02:01:02,206 adding 'colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py' 2023-03-10T02:01:02,210 adding 'colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py' 2023-03-10T02:01:02,213 adding 'colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py' 2023-03-10T02:01:02,216 adding 'colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py' 2023-03-10T02:01:02,219 adding 'colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py' 2023-03-10T02:01:02,223 adding 'colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py' 2023-03-10T02:01:02,225 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py' 2023-03-10T02:01:02,228 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py' 2023-03-10T02:01:02,231 adding 'colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py' 2023-03-10T02:01:02,235 adding 'colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py' 2023-03-10T02:01:02,241 adding 'colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py' 2023-03-10T02:01:02,247 adding 'colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py' 2023-03-10T02:01:02,250 adding 'colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py' 2023-03-10T02:01:02,252 adding 'colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py' 2023-03-10T02:01:02,255 adding 'colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py' 2023-03-10T02:01:02,258 adding 'colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py' 2023-03-10T02:01:02,261 adding 'colossalai/auto_parallel/tensor_shard/node_handler/registry.py' 2023-03-10T02:01:02,264 adding 'colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py' 2023-03-10T02:01:02,267 adding 'colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py' 2023-03-10T02:01:02,270 adding 'colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py' 2023-03-10T02:01:02,272 adding 'colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py' 2023-03-10T02:01:02,275 adding 'colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py' 2023-03-10T02:01:02,277 adding 'colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py' 2023-03-10T02:01:02,280 adding 'colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py' 2023-03-10T02:01:02,283 adding 'colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py' 2023-03-10T02:01:02,287 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py' 2023-03-10T02:01:02,291 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py' 2023-03-10T02:01:02,295 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py' 2023-03-10T02:01:02,300 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py' 2023-03-10T02:01:02,304 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py' 2023-03-10T02:01:02,307 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py' 2023-03-10T02:01:02,311 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py' 2023-03-10T02:01:02,314 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py' 2023-03-10T02:01:02,322 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py' 2023-03-10T02:01:02,326 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py' 2023-03-10T02:01:02,329 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py' 2023-03-10T02:01:02,332 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py' 2023-03-10T02:01:02,336 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py' 2023-03-10T02:01:02,340 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py' 2023-03-10T02:01:02,345 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py' 2023-03-10T02:01:02,348 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py' 2023-03-10T02:01:02,351 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py' 2023-03-10T02:01:02,354 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py' 2023-03-10T02:01:02,357 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py' 2023-03-10T02:01:02,361 adding 'colossalai/auto_parallel/tensor_shard/solver/__init__.py' 2023-03-10T02:01:02,365 adding 'colossalai/auto_parallel/tensor_shard/solver/cost_graph.py' 2023-03-10T02:01:02,368 adding 'colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py' 2023-03-10T02:01:02,374 adding 'colossalai/auto_parallel/tensor_shard/solver/solver.py' 2023-03-10T02:01:02,378 adding 'colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py' 2023-03-10T02:01:02,382 adding 'colossalai/auto_parallel/tensor_shard/utils/__init__.py' 2023-03-10T02:01:02,385 adding 'colossalai/auto_parallel/tensor_shard/utils/broadcast.py' 2023-03-10T02:01:02,389 adding 'colossalai/auto_parallel/tensor_shard/utils/factory.py' 2023-03-10T02:01:02,392 adding 'colossalai/auto_parallel/tensor_shard/utils/misc.py' 2023-03-10T02:01:02,396 adding 'colossalai/auto_parallel/tensor_shard/utils/reshape.py' 2023-03-10T02:01:02,400 adding 'colossalai/auto_parallel/tensor_shard/utils/sharding.py' 2023-03-10T02:01:02,403 adding 'colossalai/booster/__init__.py' 2023-03-10T02:01:02,406 adding 'colossalai/booster/accelerator.py' 2023-03-10T02:01:02,408 adding 'colossalai/booster/booster.py' 2023-03-10T02:01:02,411 adding 'colossalai/booster/environment_table.py' 2023-03-10T02:01:02,413 adding 'colossalai/booster/plugin.py' 2023-03-10T02:01:02,416 adding 'colossalai/booster/precision.py' 2023-03-10T02:01:02,419 adding 'colossalai/builder/__init__.py' 2023-03-10T02:01:02,422 adding 'colossalai/builder/builder.py' 2023-03-10T02:01:02,425 adding 'colossalai/cli/__init__.py' 2023-03-10T02:01:02,428 adding 'colossalai/cli/cli.py' 2023-03-10T02:01:02,431 adding 'colossalai/cli/benchmark/__init__.py' 2023-03-10T02:01:02,434 adding 'colossalai/cli/benchmark/benchmark.py' 2023-03-10T02:01:02,437 adding 'colossalai/cli/benchmark/models.py' 2023-03-10T02:01:02,440 adding 'colossalai/cli/benchmark/utils.py' 2023-03-10T02:01:02,444 adding 'colossalai/cli/check/__init__.py' 2023-03-10T02:01:02,448 adding 'colossalai/cli/check/check_installation.py' 2023-03-10T02:01:02,452 adding 'colossalai/cli/launcher/__init__.py' 2023-03-10T02:01:02,455 adding 'colossalai/cli/launcher/hostinfo.py' 2023-03-10T02:01:02,458 adding 'colossalai/cli/launcher/multinode_runner.py' 2023-03-10T02:01:02,462 adding 'colossalai/cli/launcher/run.py' 2023-03-10T02:01:02,466 adding 'colossalai/communication/__init__.py' 2023-03-10T02:01:02,470 adding 'colossalai/communication/collective.py' 2023-03-10T02:01:02,476 adding 'colossalai/communication/p2p.py' 2023-03-10T02:01:02,481 adding 'colossalai/communication/p2p_v2.py' 2023-03-10T02:01:02,483 adding 'colossalai/communication/ring.py' 2023-03-10T02:01:02,487 adding 'colossalai/communication/utils.py' 2023-03-10T02:01:02,490 adding 'colossalai/context/__init__.py' 2023-03-10T02:01:02,493 adding 'colossalai/context/config.py' 2023-03-10T02:01:02,496 adding 'colossalai/context/moe_context.py' 2023-03-10T02:01:02,503 adding 'colossalai/context/parallel_context.py' 2023-03-10T02:01:02,506 adding 'colossalai/context/parallel_mode.py' 2023-03-10T02:01:02,509 adding 'colossalai/context/singleton_meta.py' 2023-03-10T02:01:02,512 adding 'colossalai/context/process_group_initializer/__init__.py' 2023-03-10T02:01:02,515 adding 'colossalai/context/process_group_initializer/initializer_1d.py' 2023-03-10T02:01:02,518 adding 'colossalai/context/process_group_initializer/initializer_2d.py' 2023-03-10T02:01:02,522 adding 'colossalai/context/process_group_initializer/initializer_2p5d.py' 2023-03-10T02:01:02,526 adding 'colossalai/context/process_group_initializer/initializer_3d.py' 2023-03-10T02:01:02,529 adding 'colossalai/context/process_group_initializer/initializer_data.py' 2023-03-10T02:01:02,532 adding 'colossalai/context/process_group_initializer/initializer_model.py' 2023-03-10T02:01:02,534 adding 'colossalai/context/process_group_initializer/initializer_pipeline.py' 2023-03-10T02:01:02,537 adding 'colossalai/context/process_group_initializer/initializer_sequence.py' 2023-03-10T02:01:02,541 adding 'colossalai/context/process_group_initializer/initializer_tensor.py' 2023-03-10T02:01:02,543 adding 'colossalai/context/process_group_initializer/process_group_initializer.py' 2023-03-10T02:01:02,547 adding 'colossalai/context/random/__init__.py' 2023-03-10T02:01:02,550 adding 'colossalai/context/random/_helper.py' 2023-03-10T02:01:02,553 adding 'colossalai/context/random/seed_manager.py' 2023-03-10T02:01:02,556 adding 'colossalai/device/__init__.py' 2023-03-10T02:01:02,561 adding 'colossalai/device/alpha_beta_profiler.py' 2023-03-10T02:01:02,565 adding 'colossalai/device/calc_pipeline_strategy.py' 2023-03-10T02:01:02,569 adding 'colossalai/device/device_mesh.py' 2023-03-10T02:01:02,573 adding 'colossalai/engine/__init__.py' 2023-03-10T02:01:02,577 adding 'colossalai/engine/_base_engine.py' 2023-03-10T02:01:02,581 adding 'colossalai/engine/gradient_accumulation/__init__.py' 2023-03-10T02:01:02,585 adding 'colossalai/engine/gradient_accumulation/_gradient_accumulation.py' 2023-03-10T02:01:02,589 adding 'colossalai/engine/gradient_handler/__init__.py' 2023-03-10T02:01:02,591 adding 'colossalai/engine/gradient_handler/_base_gradient_handler.py' 2023-03-10T02:01:02,594 adding 'colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py' 2023-03-10T02:01:02,597 adding 'colossalai/engine/gradient_handler/_moe_gradient_handler.py' 2023-03-10T02:01:02,600 adding 'colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py' 2023-03-10T02:01:02,602 adding 'colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py' 2023-03-10T02:01:02,605 adding 'colossalai/engine/gradient_handler/_zero_gradient_handler.py' 2023-03-10T02:01:02,607 adding 'colossalai/engine/gradient_handler/utils.py' 2023-03-10T02:01:02,611 adding 'colossalai/engine/schedule/__init__.py' 2023-03-10T02:01:02,614 adding 'colossalai/engine/schedule/_base_schedule.py' 2023-03-10T02:01:02,617 adding 'colossalai/engine/schedule/_non_pipeline_schedule.py' 2023-03-10T02:01:02,627 adding 'colossalai/engine/schedule/_pipeline_schedule.py' 2023-03-10T02:01:02,631 adding 'colossalai/engine/schedule/_pipeline_schedule_v2.py' 2023-03-10T02:01:02,635 adding 'colossalai/fx/__init__.py' 2023-03-10T02:01:02,637 adding 'colossalai/fx/_compatibility.py' 2023-03-10T02:01:02,643 adding 'colossalai/fx/_meta_registrations.py' 2023-03-10T02:01:02,647 adding 'colossalai/fx/graph_module.py' 2023-03-10T02:01:02,650 adding 'colossalai/fx/proxy.py' 2023-03-10T02:01:02,653 adding 'colossalai/fx/codegen/__init__.py' 2023-03-10T02:01:02,664 adding 'colossalai/fx/codegen/activation_checkpoint_codegen.py' 2023-03-10T02:01:02,668 adding 'colossalai/fx/passes/__init__.py' 2023-03-10T02:01:02,673 adding 'colossalai/fx/passes/adding_split_node_pass.py' 2023-03-10T02:01:02,677 adding 'colossalai/fx/passes/concrete_info_prop.py' 2023-03-10T02:01:02,682 adding 'colossalai/fx/passes/meta_info_prop.py' 2023-03-10T02:01:02,688 adding 'colossalai/fx/passes/passes_for_gpt2_test.py' 2023-03-10T02:01:02,691 adding 'colossalai/fx/passes/shard_1d_pass.py' 2023-03-10T02:01:02,696 adding 'colossalai/fx/passes/split_module.py' 2023-03-10T02:01:02,700 adding 'colossalai/fx/passes/utils.py' 2023-03-10T02:01:02,704 adding 'colossalai/fx/profiler/__init__.py' 2023-03-10T02:01:02,706 adding 'colossalai/fx/profiler/constants.py' 2023-03-10T02:01:02,710 adding 'colossalai/fx/profiler/dataflow.py' 2023-03-10T02:01:02,713 adding 'colossalai/fx/profiler/memory_utils.py' 2023-03-10T02:01:02,717 adding 'colossalai/fx/profiler/opcount.py' 2023-03-10T02:01:02,722 adding 'colossalai/fx/profiler/profiler.py' 2023-03-10T02:01:02,725 adding 'colossalai/fx/profiler/shard_utils.py' 2023-03-10T02:01:02,729 adding 'colossalai/fx/profiler/tensor.py' 2023-03-10T02:01:02,732 adding 'colossalai/fx/profiler/experimental/__init__.py' 2023-03-10T02:01:02,735 adding 'colossalai/fx/profiler/experimental/constants.py' 2023-03-10T02:01:02,738 adding 'colossalai/fx/profiler/experimental/profiler.py' 2023-03-10T02:01:02,741 adding 'colossalai/fx/profiler/experimental/registry.py' 2023-03-10T02:01:02,743 adding 'colossalai/fx/profiler/experimental/shard_utils.py' 2023-03-10T02:01:02,747 adding 'colossalai/fx/profiler/experimental/profiler_function/__init__.py' 2023-03-10T02:01:02,750 adding 'colossalai/fx/profiler/experimental/profiler_function/activation_function.py' 2023-03-10T02:01:02,752 adding 'colossalai/fx/profiler/experimental/profiler_function/arithmetic.py' 2023-03-10T02:01:02,755 adding 'colossalai/fx/profiler/experimental/profiler_function/embedding.py' 2023-03-10T02:01:02,757 adding 'colossalai/fx/profiler/experimental/profiler_function/linear.py' 2023-03-10T02:01:02,760 adding 'colossalai/fx/profiler/experimental/profiler_function/normalization.py' 2023-03-10T02:01:02,762 adding 'colossalai/fx/profiler/experimental/profiler_function/pooling.py' 2023-03-10T02:01:02,765 adding 'colossalai/fx/profiler/experimental/profiler_function/python_ops.py' 2023-03-10T02:01:02,767 adding 'colossalai/fx/profiler/experimental/profiler_function/torch_ops.py' 2023-03-10T02:01:02,771 adding 'colossalai/fx/profiler/experimental/profiler_module/__init__.py' 2023-03-10T02:01:02,774 adding 'colossalai/fx/profiler/experimental/profiler_module/activation_function.py' 2023-03-10T02:01:02,777 adding 'colossalai/fx/profiler/experimental/profiler_module/attention.py' 2023-03-10T02:01:02,780 adding 'colossalai/fx/profiler/experimental/profiler_module/convolution.py' 2023-03-10T02:01:02,782 adding 'colossalai/fx/profiler/experimental/profiler_module/dropout.py' 2023-03-10T02:01:02,785 adding 'colossalai/fx/profiler/experimental/profiler_module/embedding.py' 2023-03-10T02:01:02,787 adding 'colossalai/fx/profiler/experimental/profiler_module/linear.py' 2023-03-10T02:01:02,790 adding 'colossalai/fx/profiler/experimental/profiler_module/normalization.py' 2023-03-10T02:01:02,792 adding 'colossalai/fx/profiler/experimental/profiler_module/pooling.py' 2023-03-10T02:01:02,795 adding 'colossalai/fx/profiler/experimental/profiler_module/rnn.py' 2023-03-10T02:01:02,798 adding 'colossalai/fx/profiler/experimental/profiler_module/torch_op.py' 2023-03-10T02:01:02,801 adding 'colossalai/fx/tracer/__init__.py' 2023-03-10T02:01:02,804 adding 'colossalai/fx/tracer/_meta_trace.py' 2023-03-10T02:01:02,807 adding 'colossalai/fx/tracer/_symbolic_trace.py' 2023-03-10T02:01:02,810 adding 'colossalai/fx/tracer/_tracer_utils.py' 2023-03-10T02:01:02,817 adding 'colossalai/fx/tracer/experimental.py' 2023-03-10T02:01:02,820 adding 'colossalai/fx/tracer/registry.py' 2023-03-10T02:01:02,827 adding 'colossalai/fx/tracer/tracer.py' 2023-03-10T02:01:02,831 adding 'colossalai/fx/tracer/bias_addition_patch/__init__.py' 2023-03-10T02:01:02,835 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py' 2023-03-10T02:01:02,838 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py' 2023-03-10T02:01:02,840 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py' 2023-03-10T02:01:02,843 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py' 2023-03-10T02:01:02,846 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py' 2023-03-10T02:01:02,849 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py' 2023-03-10T02:01:02,852 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py' 2023-03-10T02:01:02,855 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py' 2023-03-10T02:01:02,858 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py' 2023-03-10T02:01:02,861 adding 'colossalai/fx/tracer/meta_patch/__init__.py' 2023-03-10T02:01:02,865 adding 'colossalai/fx/tracer/meta_patch/patched_function/__init__.py' 2023-03-10T02:01:02,867 adding 'colossalai/fx/tracer/meta_patch/patched_function/activation_function.py' 2023-03-10T02:01:02,870 adding 'colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py' 2023-03-10T02:01:02,873 adding 'colossalai/fx/tracer/meta_patch/patched_function/convolution.py' 2023-03-10T02:01:02,875 adding 'colossalai/fx/tracer/meta_patch/patched_function/embedding.py' 2023-03-10T02:01:02,878 adding 'colossalai/fx/tracer/meta_patch/patched_function/normalization.py' 2023-03-10T02:01:02,880 adding 'colossalai/fx/tracer/meta_patch/patched_function/python_ops.py' 2023-03-10T02:01:02,884 adding 'colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py' 2023-03-10T02:01:02,887 adding 'colossalai/fx/tracer/meta_patch/patched_module/__init__.py' 2023-03-10T02:01:02,890 adding 'colossalai/fx/tracer/meta_patch/patched_module/activation_function.py' 2023-03-10T02:01:02,893 adding 'colossalai/fx/tracer/meta_patch/patched_module/convolution.py' 2023-03-10T02:01:02,895 adding 'colossalai/fx/tracer/meta_patch/patched_module/embedding.py' 2023-03-10T02:01:02,897 adding 'colossalai/fx/tracer/meta_patch/patched_module/linear.py' 2023-03-10T02:01:02,900 adding 'colossalai/fx/tracer/meta_patch/patched_module/normalization.py' 2023-03-10T02:01:02,903 adding 'colossalai/fx/tracer/meta_patch/patched_module/pooling.py' 2023-03-10T02:01:02,906 adding 'colossalai/fx/tracer/meta_patch/patched_module/rnn.py' 2023-03-10T02:01:02,910 adding 'colossalai/gemini/__init__.py' 2023-03-10T02:01:02,912 adding 'colossalai/gemini/gemini_context.py' 2023-03-10T02:01:02,916 adding 'colossalai/gemini/gemini_mgr.py' 2023-03-10T02:01:02,920 adding 'colossalai/gemini/placement_policy.py' 2023-03-10T02:01:02,923 adding 'colossalai/gemini/stateful_tensor.py' 2023-03-10T02:01:02,926 adding 'colossalai/gemini/stateful_tensor_mgr.py' 2023-03-10T02:01:02,930 adding 'colossalai/gemini/tensor_placement_policy.py' 2023-03-10T02:01:02,933 adding 'colossalai/gemini/tensor_utils.py' 2023-03-10T02:01:02,937 adding 'colossalai/gemini/chunk/__init__.py' 2023-03-10T02:01:02,943 adding 'colossalai/gemini/chunk/chunk.py' 2023-03-10T02:01:02,948 adding 'colossalai/gemini/chunk/manager.py' 2023-03-10T02:01:02,951 adding 'colossalai/gemini/chunk/search_utils.py' 2023-03-10T02:01:02,954 adding 'colossalai/gemini/chunk/utils.py' 2023-03-10T02:01:02,958 adding 'colossalai/gemini/memory_tracer/__init__.py' 2023-03-10T02:01:02,961 adding 'colossalai/gemini/memory_tracer/chunk_memstats_collector.py' 2023-03-10T02:01:02,964 adding 'colossalai/gemini/memory_tracer/memory_monitor.py' 2023-03-10T02:01:02,966 adding 'colossalai/gemini/memory_tracer/memory_stats.py' 2023-03-10T02:01:02,970 adding 'colossalai/gemini/memory_tracer/memstats_collector.py' 2023-03-10T02:01:02,972 adding 'colossalai/gemini/memory_tracer/param_runtime_order.py' 2023-03-10T02:01:02,975 adding 'colossalai/gemini/memory_tracer/runtime_mem_tracer.py' 2023-03-10T02:01:02,978 adding 'colossalai/gemini/memory_tracer/static_memstats_collector.py' 2023-03-10T02:01:02,981 adding 'colossalai/gemini/memory_tracer/utils.py' 2023-03-10T02:01:02,984 adding 'colossalai/gemini/ophooks/__init__.py' 2023-03-10T02:01:02,987 adding 'colossalai/gemini/ophooks/_shard_grad_ophook.py' 2023-03-10T02:01:02,989 adding 'colossalai/gemini/ophooks/_shard_param_ophook.py' 2023-03-10T02:01:02,992 adding 'colossalai/gemini/ophooks/runtime_mem_tracer_hook.py' 2023-03-10T02:01:02,996 adding 'colossalai/gemini/ophooks/utils.py' 2023-03-10T02:01:02,999 adding 'colossalai/gemini/paramhooks/__init__.py' 2023-03-10T02:01:03,002 adding 'colossalai/gemini/paramhooks/_param_hookmgr.py' 2023-03-10T02:01:03,005 adding 'colossalai/kernel/__init__.py' 2023-03-10T02:01:03,009 adding 'colossalai/kernel/cuda_native/__init__.py' 2023-03-10T02:01:03,014 adding 'colossalai/kernel/cuda_native/flash_attention.py' 2023-03-10T02:01:03,017 adding 'colossalai/kernel/cuda_native/layer_norm.py' 2023-03-10T02:01:03,022 adding 'colossalai/kernel/cuda_native/multihead_attention.py' 2023-03-10T02:01:03,025 adding 'colossalai/kernel/cuda_native/scaled_softmax.py' 2023-03-10T02:01:03,030 adding 'colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp' 2023-03-10T02:01:03,033 adding 'colossalai/kernel/cuda_native/csrc/compat.h' 2023-03-10T02:01:03,037 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.cpp' 2023-03-10T02:01:03,041 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.h' 2023-03-10T02:01:03,044 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp' 2023-03-10T02:01:03,051 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu' 2023-03-10T02:01:03,054 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda.cpp' 2023-03-10T02:01:03,060 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu' 2023-03-10T02:01:03,064 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu' 2023-03-10T02:01:03,067 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh' 2023-03-10T02:01:03,071 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu' 2023-03-10T02:01:03,076 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu' 2023-03-10T02:01:03,079 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu' 2023-03-10T02:01:03,083 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu' 2023-03-10T02:01:03,088 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp' 2023-03-10T02:01:03,091 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h' 2023-03-10T02:01:03,094 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp' 2023-03-10T02:01:03,099 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h' 2023-03-10T02:01:03,102 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu' 2023-03-10T02:01:03,105 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp' 2023-03-10T02:01:03,110 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h' 2023-03-10T02:01:03,113 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu' 2023-03-10T02:01:03,117 adding 'colossalai/kernel/cuda_native/csrc/type_shim.h' 2023-03-10T02:01:03,123 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h' 2023-03-10T02:01:03,125 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/context.h' 2023-03-10T02:01:03,128 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h' 2023-03-10T02:01:03,131 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h' 2023-03-10T02:01:03,134 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h' 2023-03-10T02:01:03,139 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h' 2023-03-10T02:01:03,142 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h' 2023-03-10T02:01:03,146 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h' 2023-03-10T02:01:03,148 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh' 2023-03-10T02:01:03,151 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h' 2023-03-10T02:01:03,154 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h' 2023-03-10T02:01:03,157 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h' 2023-03-10T02:01:03,160 adding 'colossalai/kernel/jit/__init__.py' 2023-03-10T02:01:03,163 adding 'colossalai/kernel/jit/bias_dropout_add.py' 2023-03-10T02:01:03,165 adding 'colossalai/kernel/jit/bias_gelu.py' 2023-03-10T02:01:03,168 adding 'colossalai/kernel/jit/option.py' 2023-03-10T02:01:03,173 adding 'colossalai/kernel/op_builder/__init__.py' 2023-03-10T02:01:03,178 adding 'colossalai/kernel/op_builder/builder.py' 2023-03-10T02:01:03,182 adding 'colossalai/kernel/op_builder/cpu_adam.py' 2023-03-10T02:01:03,185 adding 'colossalai/kernel/op_builder/fused_optim.py' 2023-03-10T02:01:03,188 adding 'colossalai/kernel/op_builder/layernorm.py' 2023-03-10T02:01:03,191 adding 'colossalai/kernel/op_builder/moe.py' 2023-03-10T02:01:03,193 adding 'colossalai/kernel/op_builder/multi_head_attn.py' 2023-03-10T02:01:03,196 adding 'colossalai/kernel/op_builder/scaled_masked_softmax.py' 2023-03-10T02:01:03,199 adding 'colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py' 2023-03-10T02:01:03,202 adding 'colossalai/kernel/op_builder/utils.py' 2023-03-10T02:01:03,206 adding 'colossalai/logging/__init__.py' 2023-03-10T02:01:03,210 adding 'colossalai/logging/logger.py' 2023-03-10T02:01:03,214 adding 'colossalai/nn/__init__.py' 2023-03-10T02:01:03,218 adding 'colossalai/nn/init.py' 2023-03-10T02:01:03,223 adding 'colossalai/nn/_ops/__init__.py' 2023-03-10T02:01:03,227 adding 'colossalai/nn/_ops/_utils.py' 2023-03-10T02:01:03,232 adding 'colossalai/nn/_ops/addmm.py' 2023-03-10T02:01:03,234 adding 'colossalai/nn/_ops/batch_norm.py' 2023-03-10T02:01:03,237 adding 'colossalai/nn/_ops/element_wise.py' 2023-03-10T02:01:03,242 adding 'colossalai/nn/_ops/embedding.py' 2023-03-10T02:01:03,245 adding 'colossalai/nn/_ops/embedding_bag.py' 2023-03-10T02:01:03,248 adding 'colossalai/nn/_ops/layernorm.py' 2023-03-10T02:01:03,251 adding 'colossalai/nn/_ops/linear.py' 2023-03-10T02:01:03,254 adding 'colossalai/nn/_ops/loss.py' 2023-03-10T02:01:03,257 adding 'colossalai/nn/_ops/view.py' 2023-03-10T02:01:03,260 adding 'colossalai/nn/layer/__init__.py' 2023-03-10T02:01:03,263 adding 'colossalai/nn/layer/base_layer.py' 2023-03-10T02:01:03,267 adding 'colossalai/nn/layer/colossalai_layer/__init__.py' 2023-03-10T02:01:03,269 adding 'colossalai/nn/layer/colossalai_layer/_utils.py' 2023-03-10T02:01:03,272 adding 'colossalai/nn/layer/colossalai_layer/dropout.py' 2023-03-10T02:01:03,275 adding 'colossalai/nn/layer/colossalai_layer/embedding.py' 2023-03-10T02:01:03,278 adding 'colossalai/nn/layer/colossalai_layer/linear.py' 2023-03-10T02:01:03,281 adding 'colossalai/nn/layer/colossalai_layer/normalization.py' 2023-03-10T02:01:03,284 adding 'colossalai/nn/layer/moe/__init__.py' 2023-03-10T02:01:03,288 adding 'colossalai/nn/layer/moe/_operation.py' 2023-03-10T02:01:03,291 adding 'colossalai/nn/layer/moe/experts.py' 2023-03-10T02:01:03,295 adding 'colossalai/nn/layer/moe/layers.py' 2023-03-10T02:01:03,299 adding 'colossalai/nn/layer/moe/routers.py' 2023-03-10T02:01:03,302 adding 'colossalai/nn/layer/moe/utils.py' 2023-03-10T02:01:03,306 adding 'colossalai/nn/layer/parallel_1d/__init__.py' 2023-03-10T02:01:03,309 adding 'colossalai/nn/layer/parallel_1d/_operation.py' 2023-03-10T02:01:03,312 adding 'colossalai/nn/layer/parallel_1d/_utils.py' 2023-03-10T02:01:03,321 adding 'colossalai/nn/layer/parallel_1d/layers.py' 2023-03-10T02:01:03,325 adding 'colossalai/nn/layer/parallel_2d/__init__.py' 2023-03-10T02:01:03,332 adding 'colossalai/nn/layer/parallel_2d/_operation.py' 2023-03-10T02:01:03,335 adding 'colossalai/nn/layer/parallel_2d/_utils.py' 2023-03-10T02:01:03,343 adding 'colossalai/nn/layer/parallel_2d/layers.py' 2023-03-10T02:01:03,348 adding 'colossalai/nn/layer/parallel_2p5d/__init__.py' 2023-03-10T02:01:03,355 adding 'colossalai/nn/layer/parallel_2p5d/_operation.py' 2023-03-10T02:01:03,358 adding 'colossalai/nn/layer/parallel_2p5d/_utils.py' 2023-03-10T02:01:03,367 adding 'colossalai/nn/layer/parallel_2p5d/layers.py' 2023-03-10T02:01:03,372 adding 'colossalai/nn/layer/parallel_3d/__init__.py' 2023-03-10T02:01:03,377 adding 'colossalai/nn/layer/parallel_3d/_operation.py' 2023-03-10T02:01:03,380 adding 'colossalai/nn/layer/parallel_3d/_utils.py' 2023-03-10T02:01:03,388 adding 'colossalai/nn/layer/parallel_3d/layers.py' 2023-03-10T02:01:03,393 adding 'colossalai/nn/layer/parallel_sequence/__init__.py' 2023-03-10T02:01:03,396 adding 'colossalai/nn/layer/parallel_sequence/_operation.py' 2023-03-10T02:01:03,399 adding 'colossalai/nn/layer/parallel_sequence/_utils.py' 2023-03-10T02:01:03,403 adding 'colossalai/nn/layer/parallel_sequence/layers.py' 2023-03-10T02:01:03,406 adding 'colossalai/nn/layer/utils/__init__.py' 2023-03-10T02:01:03,409 adding 'colossalai/nn/layer/utils/common.py' 2023-03-10T02:01:03,413 adding 'colossalai/nn/layer/vanilla/__init__.py' 2023-03-10T02:01:03,418 adding 'colossalai/nn/layer/vanilla/layers.py' 2023-03-10T02:01:03,422 adding 'colossalai/nn/layer/wrapper/__init__.py' 2023-03-10T02:01:03,425 adding 'colossalai/nn/layer/wrapper/pipeline_wrapper.py' 2023-03-10T02:01:03,428 adding 'colossalai/nn/loss/__init__.py' 2023-03-10T02:01:03,432 adding 'colossalai/nn/loss/loss_1d.py' 2023-03-10T02:01:03,435 adding 'colossalai/nn/loss/loss_2d.py' 2023-03-10T02:01:03,438 adding 'colossalai/nn/loss/loss_2p5d.py' 2023-03-10T02:01:03,442 adding 'colossalai/nn/loss/loss_3d.py' 2023-03-10T02:01:03,445 adding 'colossalai/nn/loss/loss_moe.py' 2023-03-10T02:01:03,448 adding 'colossalai/nn/lr_scheduler/__init__.py' 2023-03-10T02:01:03,452 adding 'colossalai/nn/lr_scheduler/cosine.py' 2023-03-10T02:01:03,455 adding 'colossalai/nn/lr_scheduler/delayed.py' 2023-03-10T02:01:03,458 adding 'colossalai/nn/lr_scheduler/linear.py' 2023-03-10T02:01:03,460 adding 'colossalai/nn/lr_scheduler/multistep.py' 2023-03-10T02:01:03,464 adding 'colossalai/nn/lr_scheduler/onecycle.py' 2023-03-10T02:01:03,466 adding 'colossalai/nn/lr_scheduler/poly.py' 2023-03-10T02:01:03,469 adding 'colossalai/nn/lr_scheduler/torch.py' 2023-03-10T02:01:03,473 adding 'colossalai/nn/metric/__init__.py' 2023-03-10T02:01:03,475 adding 'colossalai/nn/metric/_utils.py' 2023-03-10T02:01:03,478 adding 'colossalai/nn/metric/accuracy_2d.py' 2023-03-10T02:01:03,480 adding 'colossalai/nn/metric/accuracy_2p5d.py' 2023-03-10T02:01:03,483 adding 'colossalai/nn/metric/accuracy_3d.py' 2023-03-10T02:01:03,487 adding 'colossalai/nn/optimizer/__init__.py' 2023-03-10T02:01:03,489 adding 'colossalai/nn/optimizer/colossalai_optimizer.py' 2023-03-10T02:01:03,493 adding 'colossalai/nn/optimizer/cpu_adam.py' 2023-03-10T02:01:03,496 adding 'colossalai/nn/optimizer/fused_adam.py' 2023-03-10T02:01:03,500 adding 'colossalai/nn/optimizer/fused_lamb.py' 2023-03-10T02:01:03,504 adding 'colossalai/nn/optimizer/fused_sgd.py' 2023-03-10T02:01:03,506 adding 'colossalai/nn/optimizer/gemini_optimizer.py' 2023-03-10T02:01:03,510 adding 'colossalai/nn/optimizer/hybrid_adam.py' 2023-03-10T02:01:03,513 adding 'colossalai/nn/optimizer/lamb.py' 2023-03-10T02:01:03,516 adding 'colossalai/nn/optimizer/lars.py' 2023-03-10T02:01:03,519 adding 'colossalai/nn/optimizer/nvme_optimizer.py' 2023-03-10T02:01:03,524 adding 'colossalai/nn/optimizer/zero_optimizer.py' 2023-03-10T02:01:03,528 adding 'colossalai/nn/parallel/__init__.py' 2023-03-10T02:01:03,537 adding 'colossalai/nn/parallel/data_parallel.py' 2023-03-10T02:01:03,541 adding 'colossalai/nn/parallel/gemini_parallel.py' 2023-03-10T02:01:03,544 adding 'colossalai/nn/parallel/reducer.py' 2023-03-10T02:01:03,547 adding 'colossalai/nn/parallel/utils.py' 2023-03-10T02:01:03,550 adding 'colossalai/nn/parallel/zero_wrapper.py' 2023-03-10T02:01:03,554 adding 'colossalai/nn/parallel/layers/__init__.py' 2023-03-10T02:01:03,556 adding 'colossalai/nn/parallel/layers/colo_module.py' 2023-03-10T02:01:03,559 adding 'colossalai/nn/parallel/layers/embedding.py' 2023-03-10T02:01:03,561 adding 'colossalai/nn/parallel/layers/linear.py' 2023-03-10T02:01:03,564 adding 'colossalai/nn/parallel/layers/module_utils.py' 2023-03-10T02:01:03,569 adding 'colossalai/nn/parallel/layers/cache_embedding/__init__.py' 2023-03-10T02:01:03,571 adding 'colossalai/nn/parallel/layers/cache_embedding/base_embedding.py' 2023-03-10T02:01:03,579 adding 'colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py' 2023-03-10T02:01:03,583 adding 'colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py' 2023-03-10T02:01:03,586 adding 'colossalai/nn/parallel/layers/cache_embedding/copyer.py' 2023-03-10T02:01:03,589 adding 'colossalai/nn/parallel/layers/cache_embedding/embedding_config.py' 2023-03-10T02:01:03,592 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py' 2023-03-10T02:01:03,596 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py' 2023-03-10T02:01:03,599 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py' 2023-03-10T02:01:03,603 adding 'colossalai/pipeline/__init__.py' 2023-03-10T02:01:03,606 adding 'colossalai/pipeline/layer_spec.py' 2023-03-10T02:01:03,610 adding 'colossalai/pipeline/pipelinable.py' 2023-03-10T02:01:03,613 adding 'colossalai/pipeline/pipeline_process_group.py' 2023-03-10T02:01:03,617 adding 'colossalai/pipeline/utils.py' 2023-03-10T02:01:03,621 adding 'colossalai/pipeline/middleware/__init__.py' 2023-03-10T02:01:03,624 adding 'colossalai/pipeline/middleware/topo.py' 2023-03-10T02:01:03,627 adding 'colossalai/pipeline/middleware/adaptor/__init__.py' 2023-03-10T02:01:03,631 adding 'colossalai/pipeline/middleware/adaptor/fx.py' 2023-03-10T02:01:03,634 adding 'colossalai/pipeline/rpc/__init__.py' 2023-03-10T02:01:03,648 adding 'colossalai/pipeline/rpc/_pipeline_base.py' 2023-03-10T02:01:03,654 adding 'colossalai/pipeline/rpc/_pipeline_schedule.py' 2023-03-10T02:01:03,657 adding 'colossalai/pipeline/rpc/utils.py' 2023-03-10T02:01:03,661 adding 'colossalai/registry/__init__.py' 2023-03-10T02:01:03,664 adding 'colossalai/registry/registry.py' 2023-03-10T02:01:03,668 adding 'colossalai/tensor/__init__.py' 2023-03-10T02:01:03,671 adding 'colossalai/tensor/colo_parameter.py' 2023-03-10T02:01:03,676 adding 'colossalai/tensor/colo_tensor.py' 2023-03-10T02:01:03,682 adding 'colossalai/tensor/comm_spec.py' 2023-03-10T02:01:03,685 adding 'colossalai/tensor/compute_spec.py' 2023-03-10T02:01:03,687 adding 'colossalai/tensor/const.py' 2023-03-10T02:01:03,691 adding 'colossalai/tensor/dist_spec_mgr.py' 2023-03-10T02:01:03,694 adding 'colossalai/tensor/distspec.py' 2023-03-10T02:01:03,696 adding 'colossalai/tensor/op_wrapper.py' 2023-03-10T02:01:03,699 adding 'colossalai/tensor/param_op_hook.py' 2023-03-10T02:01:03,703 adding 'colossalai/tensor/process_group.py' 2023-03-10T02:01:03,712 adding 'colossalai/tensor/shape_consistency.py' 2023-03-10T02:01:03,717 adding 'colossalai/tensor/sharding_spec.py' 2023-03-10T02:01:03,720 adding 'colossalai/tensor/tensor_spec.py' 2023-03-10T02:01:03,723 adding 'colossalai/tensor/utils.py' 2023-03-10T02:01:03,727 adding 'colossalai/tensor/d_tensor/__init__.py' 2023-03-10T02:01:03,731 adding 'colossalai/tensor/d_tensor/comm_spec.py' 2023-03-10T02:01:03,734 adding 'colossalai/tensor/d_tensor/d_tensor.py' 2023-03-10T02:01:03,737 adding 'colossalai/tensor/d_tensor/layout.py' 2023-03-10T02:01:03,739 adding 'colossalai/tensor/d_tensor/misc.py' 2023-03-10T02:01:03,743 adding 'colossalai/tensor/d_tensor/sharding_spec.py' 2023-03-10T02:01:03,747 adding 'colossalai/testing/__init__.py' 2023-03-10T02:01:03,750 adding 'colossalai/testing/comparison.py' 2023-03-10T02:01:03,752 adding 'colossalai/testing/pytest_wrapper.py' 2023-03-10T02:01:03,755 adding 'colossalai/testing/random.py' 2023-03-10T02:01:03,758 adding 'colossalai/testing/utils.py' 2023-03-10T02:01:03,761 adding 'colossalai/trainer/__init__.py' 2023-03-10T02:01:03,766 adding 'colossalai/trainer/_trainer.py' 2023-03-10T02:01:03,770 adding 'colossalai/trainer/hooks/__init__.py' 2023-03-10T02:01:03,773 adding 'colossalai/trainer/hooks/_base_hook.py' 2023-03-10T02:01:03,776 adding 'colossalai/trainer/hooks/_checkpoint_hook.py' 2023-03-10T02:01:03,779 adding 'colossalai/trainer/hooks/_commons_.py' 2023-03-10T02:01:03,783 adding 'colossalai/trainer/hooks/_log_hook.py' 2023-03-10T02:01:03,786 adding 'colossalai/trainer/hooks/_lr_scheduler_hook.py' 2023-03-10T02:01:03,791 adding 'colossalai/trainer/hooks/_metric_hook.py' 2023-03-10T02:01:03,796 adding 'colossalai/utils/__init__.py' 2023-03-10T02:01:03,800 adding 'colossalai/utils/activation_checkpoint.py' 2023-03-10T02:01:03,804 adding 'colossalai/utils/checkpointing.py' 2023-03-10T02:01:03,810 adding 'colossalai/utils/common.py' 2023-03-10T02:01:03,813 adding 'colossalai/utils/cuda.py' 2023-03-10T02:01:03,816 adding 'colossalai/utils/memory.py' 2023-03-10T02:01:03,819 adding 'colossalai/utils/moe.py' 2023-03-10T02:01:03,822 adding 'colossalai/utils/timer.py' 2023-03-10T02:01:03,825 adding 'colossalai/utils/checkpoint/__init__.py' 2023-03-10T02:01:03,828 adding 'colossalai/utils/checkpoint/module_checkpoint.py' 2023-03-10T02:01:03,831 adding 'colossalai/utils/checkpoint/utils.py' 2023-03-10T02:01:03,835 adding 'colossalai/utils/checkpoint_io/__init__.py' 2023-03-10T02:01:03,837 adding 'colossalai/utils/checkpoint_io/backend.py' 2023-03-10T02:01:03,840 adding 'colossalai/utils/checkpoint_io/constant.py' 2023-03-10T02:01:03,843 adding 'colossalai/utils/checkpoint_io/convertor.py' 2023-03-10T02:01:03,847 adding 'colossalai/utils/checkpoint_io/distributed.py' 2023-03-10T02:01:03,850 adding 'colossalai/utils/checkpoint_io/io.py' 2023-03-10T02:01:03,853 adding 'colossalai/utils/checkpoint_io/meta.py' 2023-03-10T02:01:03,856 adding 'colossalai/utils/checkpoint_io/reader.py' 2023-03-10T02:01:03,860 adding 'colossalai/utils/checkpoint_io/utils.py' 2023-03-10T02:01:03,863 adding 'colossalai/utils/checkpoint_io/writer.py' 2023-03-10T02:01:03,867 adding 'colossalai/utils/data_sampler/__init__.py' 2023-03-10T02:01:03,869 adding 'colossalai/utils/data_sampler/base_sampler.py' 2023-03-10T02:01:03,873 adding 'colossalai/utils/data_sampler/data_parallel_sampler.py' 2023-03-10T02:01:03,876 adding 'colossalai/utils/model/__init__.py' 2023-03-10T02:01:03,880 adding 'colossalai/utils/model/colo_init_context.py' 2023-03-10T02:01:03,885 adding 'colossalai/utils/model/experimental.py' 2023-03-10T02:01:03,889 adding 'colossalai/utils/model/lazy_init_context.py' 2023-03-10T02:01:03,892 adding 'colossalai/utils/model/utils.py' 2023-03-10T02:01:03,896 adding 'colossalai/utils/multi_tensor_apply/__init__.py' 2023-03-10T02:01:03,898 adding 'colossalai/utils/multi_tensor_apply/multi_tensor_apply.py' 2023-03-10T02:01:03,902 adding 'colossalai/utils/profiler/__init__.py' 2023-03-10T02:01:03,904 adding 'colossalai/utils/profiler/extention.py' 2023-03-10T02:01:03,908 adding 'colossalai/utils/profiler/profiler.py' 2023-03-10T02:01:03,911 adding 'colossalai/utils/profiler/stateful_tensor_mem_extention.py' 2023-03-10T02:01:03,915 adding 'colossalai/utils/profiler/legacy/__init__.py' 2023-03-10T02:01:03,918 adding 'colossalai/utils/profiler/legacy/comm_profiler.py' 2023-03-10T02:01:03,922 adding 'colossalai/utils/profiler/legacy/pcie_profiler.py' 2023-03-10T02:01:03,925 adding 'colossalai/utils/profiler/legacy/prof_utils.py' 2023-03-10T02:01:03,928 adding 'colossalai/utils/rank_recorder/__init__.py' 2023-03-10T02:01:03,932 adding 'colossalai/utils/rank_recorder/rank_recorder.py' 2023-03-10T02:01:03,935 adding 'colossalai/utils/tensor_detector/__init__.py' 2023-03-10T02:01:03,939 adding 'colossalai/utils/tensor_detector/tensor_detector.py' 2023-03-10T02:01:03,942 adding 'colossalai/zero/__init__.py' 2023-03-10T02:01:03,946 adding 'colossalai/zero/init_ctx/__init__.py' 2023-03-10T02:01:03,950 adding 'colossalai/zero/init_ctx/init_context.py' 2023-03-10T02:01:03,954 adding 'colossalai/zero/shard_utils/__init__.py' 2023-03-10T02:01:03,956 adding 'colossalai/zero/shard_utils/base_shard_strategy.py' 2023-03-10T02:01:03,959 adding 'colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py' 2023-03-10T02:01:03,962 adding 'colossalai/zero/shard_utils/commons.py' 2023-03-10T02:01:03,964 adding 'colossalai/zero/shard_utils/tensor_shard_strategy.py' 2023-03-10T02:01:03,968 adding 'colossalai/zero/sharded_model/__init__.py' 2023-03-10T02:01:03,971 adding 'colossalai/zero/sharded_model/_utils.py' 2023-03-10T02:01:03,974 adding 'colossalai/zero/sharded_model/reduce_scatter.py' 2023-03-10T02:01:03,983 adding 'colossalai/zero/sharded_model/sharded_model_v2.py' 2023-03-10T02:01:03,986 adding 'colossalai/zero/sharded_model/utils.py' 2023-03-10T02:01:03,990 adding 'colossalai/zero/sharded_optim/__init__.py' 2023-03-10T02:01:03,994 adding 'colossalai/zero/sharded_optim/_utils.py' 2023-03-10T02:01:04,001 adding 'colossalai/zero/sharded_optim/low_level_optim.py' 2023-03-10T02:01:04,007 adding 'colossalai/zero/sharded_optim/sharded_optim_v2.py' 2023-03-10T02:01:04,011 adding 'colossalai/zero/sharded_optim/bookkeeping/__init__.py' 2023-03-10T02:01:04,014 adding 'colossalai/zero/sharded_optim/bookkeeping/base_store.py' 2023-03-10T02:01:04,016 adding 'colossalai/zero/sharded_optim/bookkeeping/bucket_store.py' 2023-03-10T02:01:04,019 adding 'colossalai/zero/sharded_optim/bookkeeping/gradient_store.py' 2023-03-10T02:01:04,022 adding 'colossalai/zero/sharded_optim/bookkeeping/parameter_store.py' 2023-03-10T02:01:04,025 adding 'colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py' 2023-03-10T02:01:04,028 adding 'colossalai/zero/sharded_param/__init__.py' 2023-03-10T02:01:04,031 adding 'colossalai/zero/sharded_param/sharded_param.py' 2023-03-10T02:01:04,034 adding 'colossalai/zero/sharded_param/sharded_tensor.py' 2023-03-10T02:01:04,037 adding 'colossalai/zero/utils/__init__.py' 2023-03-10T02:01:04,040 adding 'colossalai/zero/utils/gemini_hook.py' 2023-03-10T02:01:04,043 adding 'colossalai/zero/utils/zero_hook.py' 2023-03-10T02:01:04,050 adding 'op_builder/__init__.py' 2023-03-10T02:01:04,053 adding 'op_builder/builder.py' 2023-03-10T02:01:04,056 adding 'op_builder/cpu_adam.py' 2023-03-10T02:01:04,059 adding 'op_builder/fused_optim.py' 2023-03-10T02:01:04,061 adding 'op_builder/layernorm.py' 2023-03-10T02:01:04,064 adding 'op_builder/moe.py' 2023-03-10T02:01:04,067 adding 'op_builder/multi_head_attn.py' 2023-03-10T02:01:04,069 adding 'op_builder/scaled_masked_softmax.py' 2023-03-10T02:01:04,072 adding 'op_builder/scaled_upper_triangle_masked_softmax.py' 2023-03-10T02:01:04,076 adding 'op_builder/utils.py' 2023-03-10T02:01:04,082 adding 'colossalai-0.2.6.dist-info/LICENSE' 2023-03-10T02:01:04,087 adding 'colossalai-0.2.6.dist-info/METADATA' 2023-03-10T02:01:04,090 adding 'colossalai-0.2.6.dist-info/WHEEL' 2023-03-10T02:01:04,092 adding 'colossalai-0.2.6.dist-info/entry_points.txt' 2023-03-10T02:01:04,094 adding 'colossalai-0.2.6.dist-info/top_level.txt' 2023-03-10T02:01:04,111 adding 'colossalai-0.2.6.dist-info/RECORD' 2023-03-10T02:01:04,160 removing build/bdist.linux-armv7l/wheel 2023-03-10T02:01:04,655 Building wheel for colossalai (setup.py): finished with status 'done' 2023-03-10T02:01:04,688 Created wheel for colossalai: filename=colossalai-0.2.6-py3-none-any.whl size=890718 sha256=2f5357caa65bb726a2aadadbd876c6f53f87d5984bb016ccdfb411f9ccf13e2e 2023-03-10T02:01:04,690 Stored in directory: /tmp/pip-ephem-wheel-cache-_js2otsl/wheels/89/8e/51/12f0e5682b4c2618fd7c1648102a1f81dd87bc0f8b67e503ca 2023-03-10T02:01:04,769 Successfully built colossalai 2023-03-10T02:01:04,861 Removed build tracker: '/tmp/pip-build-tracker-5zp_mdz_'