2023-03-10T07:27:20,253 Created temporary directory: /tmp/pip-ephem-wheel-cache-9i_3ceke 2023-03-10T07:27:20,259 Created temporary directory: /tmp/pip-build-tracker-lxvge36r 2023-03-10T07:27:20,260 Initialized build tracking at /tmp/pip-build-tracker-lxvge36r 2023-03-10T07:27:20,260 Created build tracker: /tmp/pip-build-tracker-lxvge36r 2023-03-10T07:27:20,260 Entered build tracker: /tmp/pip-build-tracker-lxvge36r 2023-03-10T07:27:20,262 Created temporary directory: /tmp/pip-wheel-05b58ngb 2023-03-10T07:27:20,272 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2023-03-10T07:27:20,278 Created temporary directory: /tmp/pip-ephem-wheel-cache-_skaw1ct 2023-03-10T07:27:20,332 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2023-03-10T07:27:20,338 2 location(s) to search for versions of colossalai: 2023-03-10T07:27:20,338 * https://pypi.org/simple/colossalai/ 2023-03-10T07:27:20,338 * https://www.piwheels.org/simple/colossalai/ 2023-03-10T07:27:20,339 Fetching project page and analyzing links: https://pypi.org/simple/colossalai/ 2023-03-10T07:27:20,341 Getting page https://pypi.org/simple/colossalai/ 2023-03-10T07:27:20,344 Found index url https://pypi.org/simple 2023-03-10T07:27:20,528 Fetched page https://pypi.org/simple/colossalai/ as application/vnd.pypi.simple.v1+json 2023-03-10T07:27:20,534 Found link https://files.pythonhosted.org/packages/9c/c7/427dd2ed3769c6beb05cefdc18f8fad634673ed1472d5409afbdb768c7c3/colossalai-0.2.0.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.0 2023-03-10T07:27:20,536 Found link https://files.pythonhosted.org/packages/b1/70/cc6165ba3ce239c4b0c53c8ea6cabbfb31aa4b66dc580e85ff16d787384d/colossalai-0.2.1.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.1 2023-03-10T07:27:20,537 Found link https://files.pythonhosted.org/packages/87/d0/d82b49b722315654f6773eb458b929dba9949044a6b7cd087027c2391169/colossalai-0.2.2.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.2 2023-03-10T07:27:20,539 Found link https://files.pythonhosted.org/packages/c9/bb/9ec93ed622207b589fe4b1eddad9da74174eedc40ac306bff77ba823db92/colossalai-0.2.3.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.3 2023-03-10T07:27:20,540 Found link https://files.pythonhosted.org/packages/2c/26/4c38cbad383dfae44b9ececdea8f71d1be32c0ce80aadf883f915dd4d2fb/colossalai-0.2.4.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.4 2023-03-10T07:27:20,542 Found link https://files.pythonhosted.org/packages/15/05/38488d4a76c1c010da8d07e07ba14b15191d0e7398057a683f34e3c54dfe/colossalai-0.2.5.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.5 2023-03-10T07:27:20,543 Found link https://files.pythonhosted.org/packages/5a/a3/edf03a61814ca761d3133fae4bad80473c0f04693ef4af083787e71b6ac9/colossalai-0.2.6.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.6 2023-03-10T07:27:20,545 Found link https://files.pythonhosted.org/packages/25/7c/99446b8f26ba3a8706553868aa230dd9e9e153f024961967c6d916ddc943/colossalai-0.2.7.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.7 2023-03-10T07:27:20,546 Fetching project page and analyzing links: https://www.piwheels.org/simple/colossalai/ 2023-03-10T07:27:20,546 Getting page https://www.piwheels.org/simple/colossalai/ 2023-03-10T07:27:20,548 Found index url https://www.piwheels.org/simple 2023-03-10T07:27:20,771 Fetched page https://www.piwheels.org/simple/colossalai/ as text/html 2023-03-10T07:27:20,778 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.6-py3-none-any.whl#sha256=2f5357caa65bb726a2aadadbd876c6f53f87d5984bb016ccdfb411f9ccf13e2e (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T07:27:20,779 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.5-py3-none-any.whl#sha256=645c27a6955e81220cbf83dbc4eec810b1b271bdbd2a695bd3f1a179a2a348ff (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T07:27:20,779 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.4-py3-none-any.whl#sha256=4c6554305658e978bfa9b199fd59b284ba6fb60cf9b17002bed4982c13f17c1f (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T07:27:20,780 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.3-py3-none-any.whl#sha256=e5c1d7c77bd0eb9c0b3cea5c33050a759bb29fcd128ef0f6459bec9e69f4969c (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T07:27:20,780 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.2-py3-none-any.whl#sha256=272ef339dfcdc089fc4e150a9ed2377e0998b1b3e5061b8760add23228609122 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T07:27:20,781 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.1-py3-none-any.whl#sha256=fba9ebaec6b5f43d7e2424a858cae18f765cf95d36956496e4476f23f36d97b0 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T07:27:20,781 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.0-py3-none-any.whl#sha256=707a78c4c8d4b96171adc704e95bbdc387fc1dee4f612a8500f0a562c4454dc7 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-03-10T07:27:20,782 Skipping link: not a file: https://www.piwheels.org/simple/colossalai/ 2023-03-10T07:27:20,782 Skipping link: not a file: https://pypi.org/simple/colossalai/ 2023-03-10T07:27:20,817 Given no hashes to check 1 links for project 'colossalai': discarding no candidates 2023-03-10T07:27:20,847 Collecting colossalai==0.2.7 2023-03-10T07:27:20,851 Created temporary directory: /tmp/pip-unpack-ltz_sjqz 2023-03-10T07:27:21,098 Downloading colossalai-0.2.7.tar.gz (686 kB) 2023-03-10T07:27:24,545 Added colossalai==0.2.7 from https://files.pythonhosted.org/packages/25/7c/99446b8f26ba3a8706553868aa230dd9e9e153f024961967c6d916ddc943/colossalai-0.2.7.tar.gz to build tracker '/tmp/pip-build-tracker-lxvge36r' 2023-03-10T07:27:24,548 Running setup.py (path:/tmp/pip-wheel-05b58ngb/colossalai_a9b21d6d868145d583babe7311c4b7e2/setup.py) egg_info for package colossalai 2023-03-10T07:27:24,549 Created temporary directory: /tmp/pip-pip-egg-info-1mkzw5nw 2023-03-10T07:27:24,550 Preparing metadata (setup.py): started 2023-03-10T07:27:24,552 Running command python setup.py egg_info 2023-03-10T07:27:26,691 running egg_info 2023-03-10T07:27:26,693 creating /tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info 2023-03-10T07:27:26,748 writing /tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/PKG-INFO 2023-03-10T07:27:26,754 writing dependency_links to /tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/dependency_links.txt 2023-03-10T07:27:26,758 writing entry points to /tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/entry_points.txt 2023-03-10T07:27:26,762 writing requirements to /tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/requires.txt 2023-03-10T07:27:26,765 writing top-level names to /tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/top_level.txt 2023-03-10T07:27:26,768 writing manifest file '/tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/SOURCES.txt' 2023-03-10T07:27:27,144 reading manifest file '/tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/SOURCES.txt' 2023-03-10T07:27:27,148 reading manifest template 'MANIFEST.in' 2023-03-10T07:27:29,187 warning: no files found matching '*.tr' under directory 'colossalai' 2023-03-10T07:27:30,205 warning: no files found matching '*.cc' under directory 'colossalai' 2023-03-10T07:27:30,715 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-03-10T07:27:30,733 adding license file 'LICENSE' 2023-03-10T07:27:30,800 writing manifest file '/tmp/pip-pip-egg-info-1mkzw5nw/colossalai.egg-info/SOURCES.txt' 2023-03-10T07:27:30,921 Preparing metadata (setup.py): finished with status 'done' 2023-03-10T07:27:30,938 Source in /tmp/pip-wheel-05b58ngb/colossalai_a9b21d6d868145d583babe7311c4b7e2 has version 0.2.7, which satisfies requirement colossalai==0.2.7 from https://files.pythonhosted.org/packages/25/7c/99446b8f26ba3a8706553868aa230dd9e9e153f024961967c6d916ddc943/colossalai-0.2.7.tar.gz 2023-03-10T07:27:30,940 Removed colossalai==0.2.7 from https://files.pythonhosted.org/packages/25/7c/99446b8f26ba3a8706553868aa230dd9e9e153f024961967c6d916ddc943/colossalai-0.2.7.tar.gz from build tracker '/tmp/pip-build-tracker-lxvge36r' 2023-03-10T07:27:30,959 Created temporary directory: /tmp/pip-unpack-mq5nsvre 2023-03-10T07:27:30,960 Building wheels for collected packages: colossalai 2023-03-10T07:27:30,969 Created temporary directory: /tmp/pip-wheel-c5zr2w2o 2023-03-10T07:27:30,970 Building wheel for colossalai (setup.py): started 2023-03-10T07:27:30,972 Destination directory: /tmp/pip-wheel-c5zr2w2o 2023-03-10T07:27:30,972 Running command python setup.py bdist_wheel 2023-03-10T07:27:33,058 running bdist_wheel 2023-03-10T07:27:33,777 running build 2023-03-10T07:27:33,777 running build_py 2023-03-10T07:27:33,842 creating build 2023-03-10T07:27:33,843 creating build/lib 2023-03-10T07:27:33,845 creating build/lib/colossalai 2023-03-10T07:27:33,848 copying colossalai/global_variables.py -> build/lib/colossalai 2023-03-10T07:27:33,852 copying colossalai/version.py -> build/lib/colossalai 2023-03-10T07:27:33,855 copying colossalai/__init__.py -> build/lib/colossalai 2023-03-10T07:27:33,859 copying colossalai/core.py -> build/lib/colossalai 2023-03-10T07:27:33,863 copying colossalai/constants.py -> build/lib/colossalai 2023-03-10T07:27:33,867 copying colossalai/initialize.py -> build/lib/colossalai 2023-03-10T07:27:33,875 creating build/lib/op_builder 2023-03-10T07:27:33,877 copying op_builder/scaled_upper_triangle_masked_softmax.py -> build/lib/op_builder 2023-03-10T07:27:33,880 copying op_builder/fused_optim.py -> build/lib/op_builder 2023-03-10T07:27:33,884 copying op_builder/cpu_adam.py -> build/lib/op_builder 2023-03-10T07:27:33,887 copying op_builder/multi_head_attn.py -> build/lib/op_builder 2023-03-10T07:27:33,890 copying op_builder/moe.py -> build/lib/op_builder 2023-03-10T07:27:33,894 copying op_builder/utils.py -> build/lib/op_builder 2023-03-10T07:27:33,898 copying op_builder/__init__.py -> build/lib/op_builder 2023-03-10T07:27:33,901 copying op_builder/builder.py -> build/lib/op_builder 2023-03-10T07:27:33,905 copying op_builder/scaled_masked_softmax.py -> build/lib/op_builder 2023-03-10T07:27:33,909 copying op_builder/layernorm.py -> build/lib/op_builder 2023-03-10T07:27:33,914 creating build/lib/colossalai/logging 2023-03-10T07:27:33,916 copying colossalai/logging/logger.py -> build/lib/colossalai/logging 2023-03-10T07:27:33,921 copying colossalai/logging/__init__.py -> build/lib/colossalai/logging 2023-03-10T07:27:33,926 creating build/lib/colossalai/cli 2023-03-10T07:27:33,927 copying colossalai/cli/cli.py -> build/lib/colossalai/cli 2023-03-10T07:27:33,932 copying colossalai/cli/__init__.py -> build/lib/colossalai/cli 2023-03-10T07:27:33,936 creating build/lib/colossalai/builder 2023-03-10T07:27:33,938 copying colossalai/builder/__init__.py -> build/lib/colossalai/builder 2023-03-10T07:27:33,942 copying colossalai/builder/builder.py -> build/lib/colossalai/builder 2023-03-10T07:27:33,948 creating build/lib/colossalai/booster 2023-03-10T07:27:33,950 copying colossalai/booster/precision.py -> build/lib/colossalai/booster 2023-03-10T07:27:33,953 copying colossalai/booster/plugin.py -> build/lib/colossalai/booster 2023-03-10T07:27:33,957 copying colossalai/booster/environment_table.py -> build/lib/colossalai/booster 2023-03-10T07:27:33,961 copying colossalai/booster/booster.py -> build/lib/colossalai/booster 2023-03-10T07:27:33,964 copying colossalai/booster/accelerator.py -> build/lib/colossalai/booster 2023-03-10T07:27:33,968 copying colossalai/booster/__init__.py -> build/lib/colossalai/booster 2023-03-10T07:27:33,973 creating build/lib/colossalai/auto_parallel 2023-03-10T07:27:33,975 copying colossalai/auto_parallel/__init__.py -> build/lib/colossalai/auto_parallel 2023-03-10T07:27:33,980 creating build/lib/colossalai/nn 2023-03-10T07:27:33,982 copying colossalai/nn/init.py -> build/lib/colossalai/nn 2023-03-10T07:27:33,987 copying colossalai/nn/__init__.py -> build/lib/colossalai/nn 2023-03-10T07:27:33,991 creating build/lib/colossalai/device 2023-03-10T07:27:33,993 copying colossalai/device/alpha_beta_profiler.py -> build/lib/colossalai/device 2023-03-10T07:27:33,999 copying colossalai/device/calc_pipeline_strategy.py -> build/lib/colossalai/device 2023-03-10T07:27:34,003 copying colossalai/device/device_mesh.py -> build/lib/colossalai/device 2023-03-10T07:27:34,008 copying colossalai/device/__init__.py -> build/lib/colossalai/device 2023-03-10T07:27:34,013 creating build/lib/colossalai/registry 2023-03-10T07:27:34,014 copying colossalai/registry/registry.py -> build/lib/colossalai/registry 2023-03-10T07:27:34,019 copying colossalai/registry/__init__.py -> build/lib/colossalai/registry 2023-03-10T07:27:34,023 creating build/lib/colossalai/trainer 2023-03-10T07:27:34,025 copying colossalai/trainer/_trainer.py -> build/lib/colossalai/trainer 2023-03-10T07:27:34,030 copying colossalai/trainer/__init__.py -> build/lib/colossalai/trainer 2023-03-10T07:27:34,035 creating build/lib/colossalai/testing 2023-03-10T07:27:34,038 copying colossalai/testing/pytest_wrapper.py -> build/lib/colossalai/testing 2023-03-10T07:27:34,042 copying colossalai/testing/random.py -> build/lib/colossalai/testing 2023-03-10T07:27:34,045 copying colossalai/testing/utils.py -> build/lib/colossalai/testing 2023-03-10T07:27:34,049 copying colossalai/testing/__init__.py -> build/lib/colossalai/testing 2023-03-10T07:27:34,053 copying colossalai/testing/comparison.py -> build/lib/colossalai/testing 2023-03-10T07:27:34,058 creating build/lib/colossalai/gemini 2023-03-10T07:27:34,060 copying colossalai/gemini/stateful_tensor_mgr.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,065 copying colossalai/gemini/gemini_mgr.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,069 copying colossalai/gemini/placement_policy.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,074 copying colossalai/gemini/tensor_utils.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,078 copying colossalai/gemini/stateful_tensor.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,082 copying colossalai/gemini/__init__.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,086 copying colossalai/gemini/gemini_context.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,089 copying colossalai/gemini/tensor_placement_policy.py -> build/lib/colossalai/gemini 2023-03-10T07:27:34,095 creating build/lib/colossalai/communication 2023-03-10T07:27:34,097 copying colossalai/communication/ring.py -> build/lib/colossalai/communication 2023-03-10T07:27:34,101 copying colossalai/communication/collective.py -> build/lib/colossalai/communication 2023-03-10T07:27:34,105 copying colossalai/communication/p2p.py -> build/lib/colossalai/communication 2023-03-10T07:27:34,110 copying colossalai/communication/utils.py -> build/lib/colossalai/communication 2023-03-10T07:27:34,114 copying colossalai/communication/__init__.py -> build/lib/colossalai/communication 2023-03-10T07:27:34,118 copying colossalai/communication/p2p_v2.py -> build/lib/colossalai/communication 2023-03-10T07:27:34,123 creating build/lib/colossalai/amp 2023-03-10T07:27:34,125 copying colossalai/amp/__init__.py -> build/lib/colossalai/amp 2023-03-10T07:27:34,129 copying colossalai/amp/amp_type.py -> build/lib/colossalai/amp 2023-03-10T07:27:34,133 creating build/lib/colossalai/fx 2023-03-10T07:27:34,135 copying colossalai/fx/_compatibility.py -> build/lib/colossalai/fx 2023-03-10T07:27:34,139 copying colossalai/fx/_meta_registrations.py -> build/lib/colossalai/fx 2023-03-10T07:27:34,144 copying colossalai/fx/__init__.py -> build/lib/colossalai/fx 2023-03-10T07:27:34,147 copying colossalai/fx/proxy.py -> build/lib/colossalai/fx 2023-03-10T07:27:34,151 copying colossalai/fx/graph_module.py -> build/lib/colossalai/fx 2023-03-10T07:27:34,158 creating build/lib/colossalai/utils 2023-03-10T07:27:34,160 copying colossalai/utils/cuda.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,164 copying colossalai/utils/common.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,169 copying colossalai/utils/checkpointing.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,173 copying colossalai/utils/moe.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,177 copying colossalai/utils/__init__.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,181 copying colossalai/utils/memory.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,185 copying colossalai/utils/activation_checkpoint.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,190 copying colossalai/utils/timer.py -> build/lib/colossalai/utils 2023-03-10T07:27:34,197 creating build/lib/colossalai/tensor 2023-03-10T07:27:34,199 copying colossalai/tensor/param_op_hook.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,203 copying colossalai/tensor/shape_consistency.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,210 copying colossalai/tensor/comm_spec.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,215 copying colossalai/tensor/colo_parameter.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,219 copying colossalai/tensor/sharding_spec.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,224 copying colossalai/tensor/compute_spec.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,228 copying colossalai/tensor/const.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,231 copying colossalai/tensor/utils.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,235 copying colossalai/tensor/__init__.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,239 copying colossalai/tensor/colo_tensor.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,244 copying colossalai/tensor/tensor_spec.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,248 copying colossalai/tensor/process_group.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,252 copying colossalai/tensor/op_wrapper.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,256 copying colossalai/tensor/distspec.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,260 copying colossalai/tensor/dist_spec_mgr.py -> build/lib/colossalai/tensor 2023-03-10T07:27:34,265 creating build/lib/colossalai/kernel 2023-03-10T07:27:34,267 copying colossalai/kernel/__init__.py -> build/lib/colossalai/kernel 2023-03-10T07:27:34,272 creating build/lib/colossalai/pipeline 2023-03-10T07:27:34,274 copying colossalai/pipeline/pipeline_process_group.py -> build/lib/colossalai/pipeline 2023-03-10T07:27:34,279 copying colossalai/pipeline/utils.py -> build/lib/colossalai/pipeline 2023-03-10T07:27:34,283 copying colossalai/pipeline/__init__.py -> build/lib/colossalai/pipeline 2023-03-10T07:27:34,287 copying colossalai/pipeline/pipelinable.py -> build/lib/colossalai/pipeline 2023-03-10T07:27:34,291 copying colossalai/pipeline/layer_spec.py -> build/lib/colossalai/pipeline 2023-03-10T07:27:34,296 creating build/lib/colossalai/engine 2023-03-10T07:27:34,297 copying colossalai/engine/_base_engine.py -> build/lib/colossalai/engine 2023-03-10T07:27:34,302 copying colossalai/engine/__init__.py -> build/lib/colossalai/engine 2023-03-10T07:27:34,306 creating build/lib/colossalai/_C 2023-03-10T07:27:34,307 copying colossalai/_C/__init__.py -> build/lib/colossalai/_C 2023-03-10T07:27:34,312 creating build/lib/colossalai/context 2023-03-10T07:27:34,314 copying colossalai/context/moe_context.py -> build/lib/colossalai/context 2023-03-10T07:27:34,318 copying colossalai/context/singleton_meta.py -> build/lib/colossalai/context 2023-03-10T07:27:34,322 copying colossalai/context/parallel_mode.py -> build/lib/colossalai/context 2023-03-10T07:27:34,325 copying colossalai/context/config.py -> build/lib/colossalai/context 2023-03-10T07:27:34,328 copying colossalai/context/__init__.py -> build/lib/colossalai/context 2023-03-10T07:27:34,331 copying colossalai/context/parallel_context.py -> build/lib/colossalai/context 2023-03-10T07:27:34,338 creating build/lib/colossalai/zero 2023-03-10T07:27:34,340 copying colossalai/zero/__init__.py -> build/lib/colossalai/zero 2023-03-10T07:27:34,345 creating build/lib/colossalai/cli/launcher 2023-03-10T07:27:34,346 copying colossalai/cli/launcher/run.py -> build/lib/colossalai/cli/launcher 2023-03-10T07:27:34,351 copying colossalai/cli/launcher/multinode_runner.py -> build/lib/colossalai/cli/launcher 2023-03-10T07:27:34,355 copying colossalai/cli/launcher/__init__.py -> build/lib/colossalai/cli/launcher 2023-03-10T07:27:34,359 copying colossalai/cli/launcher/hostinfo.py -> build/lib/colossalai/cli/launcher 2023-03-10T07:27:34,363 creating build/lib/colossalai/cli/check 2023-03-10T07:27:34,365 copying colossalai/cli/check/check_installation.py -> build/lib/colossalai/cli/check 2023-03-10T07:27:34,369 copying colossalai/cli/check/__init__.py -> build/lib/colossalai/cli/check 2023-03-10T07:27:34,374 creating build/lib/colossalai/cli/benchmark 2023-03-10T07:27:34,375 copying colossalai/cli/benchmark/benchmark.py -> build/lib/colossalai/cli/benchmark 2023-03-10T07:27:34,380 copying colossalai/cli/benchmark/models.py -> build/lib/colossalai/cli/benchmark 2023-03-10T07:27:34,383 copying colossalai/cli/benchmark/utils.py -> build/lib/colossalai/cli/benchmark 2023-03-10T07:27:34,387 copying colossalai/cli/benchmark/__init__.py -> build/lib/colossalai/cli/benchmark 2023-03-10T07:27:34,392 creating build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:34,394 copying colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:34,399 copying colossalai/auto_parallel/tensor_shard/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:34,401 copying colossalai/auto_parallel/tensor_shard/options.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:34,405 copying colossalai/auto_parallel/tensor_shard/constants.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:34,408 copying colossalai/auto_parallel/tensor_shard/initialize.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:34,415 creating build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:34,417 copying colossalai/auto_parallel/meta_profiler/registry.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:34,421 copying colossalai/auto_parallel/meta_profiler/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:34,424 copying colossalai/auto_parallel/meta_profiler/metainfo.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:34,428 copying colossalai/auto_parallel/meta_profiler/constants.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:34,433 creating build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:34,434 copying colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:34,440 copying colossalai/auto_parallel/checkpoint/operation.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:34,444 copying colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:34,448 copying colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:34,451 copying colossalai/auto_parallel/checkpoint/__init__.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:34,455 copying colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:34,460 creating build/lib/colossalai/auto_parallel/passes 2023-03-10T07:27:34,462 copying colossalai/auto_parallel/passes/meta_info_prop.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T07:27:34,466 copying colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T07:27:34,470 copying colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T07:27:34,474 copying colossalai/auto_parallel/passes/__init__.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T07:27:34,477 copying colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T07:27:34,482 copying colossalai/auto_parallel/passes/constants.py -> build/lib/colossalai/auto_parallel/passes 2023-03-10T07:27:34,486 creating build/lib/colossalai/auto_parallel/pipeline_shard 2023-03-10T07:27:34,489 copying colossalai/auto_parallel/pipeline_shard/__init__.py -> build/lib/colossalai/auto_parallel/pipeline_shard 2023-03-10T07:27:34,493 creating build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:34,495 copying colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:34,499 copying colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:34,503 copying colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:34,507 copying colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:34,512 copying colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:34,515 copying colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:34,523 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,525 copying colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,529 copying colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,533 copying colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,536 copying colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,540 copying colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,543 copying colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,547 copying colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,552 copying colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,556 copying colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,559 copying colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,563 copying colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,566 copying colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,571 copying colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,574 copying colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,578 copying colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,582 copying colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,587 copying colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,590 copying colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,594 copying colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,598 copying colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,601 copying colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,605 copying colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,609 copying colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,612 copying colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,616 copying colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,620 copying colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,623 copying colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:34,628 creating build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:34,630 copying colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:34,634 copying colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:34,638 copying colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:34,641 copying colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:34,646 copying colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:34,661 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,663 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,667 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,671 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,676 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,680 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,684 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,688 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,691 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,695 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,699 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,703 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,708 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,712 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,716 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,721 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,725 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,729 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,733 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,736 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:34,745 creating build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,747 copying colossalai/auto_parallel/meta_profiler/meta_registry/where.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,750 copying colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,756 copying colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,759 copying colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,763 copying colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,767 copying colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,771 copying colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,774 copying colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,778 copying colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,781 copying colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,785 copying colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:34,790 creating build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,792 copying colossalai/nn/_ops/_utils.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,797 copying colossalai/nn/_ops/element_wise.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,802 copying colossalai/nn/_ops/linear.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,806 copying colossalai/nn/_ops/embedding_bag.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,810 copying colossalai/nn/_ops/view.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,814 copying colossalai/nn/_ops/__init__.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,817 copying colossalai/nn/_ops/embedding.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,822 copying colossalai/nn/_ops/loss.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,826 copying colossalai/nn/_ops/addmm.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,829 copying colossalai/nn/_ops/batch_norm.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,833 copying colossalai/nn/_ops/layernorm.py -> build/lib/colossalai/nn/_ops 2023-03-10T07:27:34,838 creating build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,840 copying colossalai/nn/lr_scheduler/onecycle.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,845 copying colossalai/nn/lr_scheduler/linear.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,848 copying colossalai/nn/lr_scheduler/torch.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,852 copying colossalai/nn/lr_scheduler/poly.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,855 copying colossalai/nn/lr_scheduler/__init__.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,859 copying colossalai/nn/lr_scheduler/multistep.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,862 copying colossalai/nn/lr_scheduler/cosine.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,867 copying colossalai/nn/lr_scheduler/delayed.py -> build/lib/colossalai/nn/lr_scheduler 2023-03-10T07:27:34,873 creating build/lib/colossalai/nn/parallel 2023-03-10T07:27:34,875 copying colossalai/nn/parallel/reducer.py -> build/lib/colossalai/nn/parallel 2023-03-10T07:27:34,879 copying colossalai/nn/parallel/zero_wrapper.py -> build/lib/colossalai/nn/parallel 2023-03-10T07:27:34,883 copying colossalai/nn/parallel/data_parallel.py -> build/lib/colossalai/nn/parallel 2023-03-10T07:27:34,889 copying colossalai/nn/parallel/utils.py -> build/lib/colossalai/nn/parallel 2023-03-10T07:27:34,893 copying colossalai/nn/parallel/__init__.py -> build/lib/colossalai/nn/parallel 2023-03-10T07:27:34,896 copying colossalai/nn/parallel/gemini_parallel.py -> build/lib/colossalai/nn/parallel 2023-03-10T07:27:34,902 creating build/lib/colossalai/nn/loss 2023-03-10T07:27:34,905 copying colossalai/nn/loss/loss_moe.py -> build/lib/colossalai/nn/loss 2023-03-10T07:27:34,910 copying colossalai/nn/loss/loss_2p5d.py -> build/lib/colossalai/nn/loss 2023-03-10T07:27:34,915 copying colossalai/nn/loss/loss_1d.py -> build/lib/colossalai/nn/loss 2023-03-10T07:27:34,919 copying colossalai/nn/loss/__init__.py -> build/lib/colossalai/nn/loss 2023-03-10T07:27:34,923 copying colossalai/nn/loss/loss_3d.py -> build/lib/colossalai/nn/loss 2023-03-10T07:27:34,927 copying colossalai/nn/loss/loss_2d.py -> build/lib/colossalai/nn/loss 2023-03-10T07:27:34,933 creating build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,935 copying colossalai/nn/optimizer/lars.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,939 copying colossalai/nn/optimizer/fused_sgd.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,943 copying colossalai/nn/optimizer/gemini_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,948 copying colossalai/nn/optimizer/lamb.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,954 copying colossalai/nn/optimizer/cpu_adam.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,959 copying colossalai/nn/optimizer/colossalai_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,962 copying colossalai/nn/optimizer/__init__.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,966 copying colossalai/nn/optimizer/zero_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,971 copying colossalai/nn/optimizer/fused_lamb.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,976 copying colossalai/nn/optimizer/hybrid_adam.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,980 copying colossalai/nn/optimizer/nvme_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,984 copying colossalai/nn/optimizer/fused_adam.py -> build/lib/colossalai/nn/optimizer 2023-03-10T07:27:34,991 creating build/lib/colossalai/nn/metric 2023-03-10T07:27:34,993 copying colossalai/nn/metric/_utils.py -> build/lib/colossalai/nn/metric 2023-03-10T07:27:34,998 copying colossalai/nn/metric/accuracy_3d.py -> build/lib/colossalai/nn/metric 2023-03-10T07:27:35,003 copying colossalai/nn/metric/__init__.py -> build/lib/colossalai/nn/metric 2023-03-10T07:27:35,006 copying colossalai/nn/metric/accuracy_2d.py -> build/lib/colossalai/nn/metric 2023-03-10T07:27:35,010 copying colossalai/nn/metric/accuracy_2p5d.py -> build/lib/colossalai/nn/metric 2023-03-10T07:27:35,015 creating build/lib/colossalai/nn/layer 2023-03-10T07:27:35,017 copying colossalai/nn/layer/base_layer.py -> build/lib/colossalai/nn/layer 2023-03-10T07:27:35,021 copying colossalai/nn/layer/__init__.py -> build/lib/colossalai/nn/layer 2023-03-10T07:27:35,026 creating build/lib/colossalai/nn/parallel/layers 2023-03-10T07:27:35,028 copying colossalai/nn/parallel/layers/linear.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T07:27:35,032 copying colossalai/nn/parallel/layers/colo_module.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T07:27:35,035 copying colossalai/nn/parallel/layers/__init__.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T07:27:35,039 copying colossalai/nn/parallel/layers/embedding.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T07:27:35,043 copying colossalai/nn/parallel/layers/module_utils.py -> build/lib/colossalai/nn/parallel/layers 2023-03-10T07:27:35,050 creating build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,052 copying colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,056 copying colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,062 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,066 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,071 copying colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,075 copying colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,079 copying colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,083 copying colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,086 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:35,093 creating build/lib/colossalai/nn/layer/moe 2023-03-10T07:27:35,095 copying colossalai/nn/layer/moe/routers.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T07:27:35,100 copying colossalai/nn/layer/moe/experts.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T07:27:35,105 copying colossalai/nn/layer/moe/layers.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T07:27:35,110 copying colossalai/nn/layer/moe/utils.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T07:27:35,114 copying colossalai/nn/layer/moe/_operation.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T07:27:35,119 copying colossalai/nn/layer/moe/__init__.py -> build/lib/colossalai/nn/layer/moe 2023-03-10T07:27:35,125 creating build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:35,128 copying colossalai/nn/layer/parallel_2d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:35,132 copying colossalai/nn/layer/parallel_2d/layers.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:35,140 copying colossalai/nn/layer/parallel_2d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:35,147 copying colossalai/nn/layer/parallel_2d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:35,152 creating build/lib/colossalai/nn/layer/vanilla 2023-03-10T07:27:35,154 copying colossalai/nn/layer/vanilla/layers.py -> build/lib/colossalai/nn/layer/vanilla 2023-03-10T07:27:35,159 copying colossalai/nn/layer/vanilla/__init__.py -> build/lib/colossalai/nn/layer/vanilla 2023-03-10T07:27:35,164 creating build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:35,166 copying colossalai/nn/layer/parallel_2p5d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:35,170 copying colossalai/nn/layer/parallel_2p5d/layers.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:35,177 copying colossalai/nn/layer/parallel_2p5d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:35,183 copying colossalai/nn/layer/parallel_2p5d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:35,188 creating build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:35,190 copying colossalai/nn/layer/parallel_sequence/_utils.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:35,194 copying colossalai/nn/layer/parallel_sequence/layers.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:35,199 copying colossalai/nn/layer/parallel_sequence/_operation.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:35,203 copying colossalai/nn/layer/parallel_sequence/__init__.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:35,207 creating build/lib/colossalai/nn/layer/utils 2023-03-10T07:27:35,209 copying colossalai/nn/layer/utils/common.py -> build/lib/colossalai/nn/layer/utils 2023-03-10T07:27:35,213 copying colossalai/nn/layer/utils/__init__.py -> build/lib/colossalai/nn/layer/utils 2023-03-10T07:27:35,218 creating build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:35,220 copying colossalai/nn/layer/parallel_1d/_utils.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:35,225 copying colossalai/nn/layer/parallel_1d/layers.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:35,231 copying colossalai/nn/layer/parallel_1d/_operation.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:35,234 copying colossalai/nn/layer/parallel_1d/__init__.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:35,240 creating build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:35,242 copying colossalai/nn/layer/colossalai_layer/normalization.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:35,246 copying colossalai/nn/layer/colossalai_layer/_utils.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:35,250 copying colossalai/nn/layer/colossalai_layer/linear.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:35,254 copying colossalai/nn/layer/colossalai_layer/dropout.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:35,259 copying colossalai/nn/layer/colossalai_layer/__init__.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:35,263 copying colossalai/nn/layer/colossalai_layer/embedding.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:35,269 creating build/lib/colossalai/nn/layer/wrapper 2023-03-10T07:27:35,272 copying colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/lib/colossalai/nn/layer/wrapper 2023-03-10T07:27:35,276 copying colossalai/nn/layer/wrapper/__init__.py -> build/lib/colossalai/nn/layer/wrapper 2023-03-10T07:27:35,282 creating build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:35,285 copying colossalai/nn/layer/parallel_3d/_utils.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:35,290 copying colossalai/nn/layer/parallel_3d/layers.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:35,297 copying colossalai/nn/layer/parallel_3d/_operation.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:35,303 copying colossalai/nn/layer/parallel_3d/__init__.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:35,308 creating build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,310 copying colossalai/trainer/hooks/_checkpoint_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,314 copying colossalai/trainer/hooks/_lr_scheduler_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,318 copying colossalai/trainer/hooks/_commons_.py -> build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,322 copying colossalai/trainer/hooks/_metric_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,327 copying colossalai/trainer/hooks/__init__.py -> build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,331 copying colossalai/trainer/hooks/_base_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,335 copying colossalai/trainer/hooks/_log_hook.py -> build/lib/colossalai/trainer/hooks 2023-03-10T07:27:35,341 creating build/lib/colossalai/gemini/chunk 2023-03-10T07:27:35,343 copying colossalai/gemini/chunk/manager.py -> build/lib/colossalai/gemini/chunk 2023-03-10T07:27:35,348 copying colossalai/gemini/chunk/search_utils.py -> build/lib/colossalai/gemini/chunk 2023-03-10T07:27:35,374 copying colossalai/gemini/chunk/utils.py -> build/lib/colossalai/gemini/chunk 2023-03-10T07:27:35,377 copying colossalai/gemini/chunk/__init__.py -> build/lib/colossalai/gemini/chunk 2023-03-10T07:27:35,381 copying colossalai/gemini/chunk/chunk.py -> build/lib/colossalai/gemini/chunk 2023-03-10T07:27:35,389 creating build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,391 copying colossalai/gemini/memory_tracer/static_memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,395 copying colossalai/gemini/memory_tracer/chunk_memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,399 copying colossalai/gemini/memory_tracer/param_runtime_order.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,403 copying colossalai/gemini/memory_tracer/utils.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,407 copying colossalai/gemini/memory_tracer/memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,411 copying colossalai/gemini/memory_tracer/memory_monitor.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,415 copying colossalai/gemini/memory_tracer/__init__.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,419 copying colossalai/gemini/memory_tracer/runtime_mem_tracer.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,423 copying colossalai/gemini/memory_tracer/memory_stats.py -> build/lib/colossalai/gemini/memory_tracer 2023-03-10T07:27:35,429 creating build/lib/colossalai/gemini/paramhooks 2023-03-10T07:27:35,431 copying colossalai/gemini/paramhooks/_param_hookmgr.py -> build/lib/colossalai/gemini/paramhooks 2023-03-10T07:27:35,434 copying colossalai/gemini/paramhooks/__init__.py -> build/lib/colossalai/gemini/paramhooks 2023-03-10T07:27:35,440 creating build/lib/colossalai/gemini/ophooks 2023-03-10T07:27:35,442 copying colossalai/gemini/ophooks/_shard_param_ophook.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T07:27:35,446 copying colossalai/gemini/ophooks/_shard_grad_ophook.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T07:27:35,450 copying colossalai/gemini/ophooks/utils.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T07:27:35,454 copying colossalai/gemini/ophooks/__init__.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T07:27:35,458 copying colossalai/gemini/ophooks/runtime_mem_tracer_hook.py -> build/lib/colossalai/gemini/ophooks 2023-03-10T07:27:35,464 creating build/lib/colossalai/amp/torch_amp 2023-03-10T07:27:35,466 copying colossalai/amp/torch_amp/_grad_scaler.py -> build/lib/colossalai/amp/torch_amp 2023-03-10T07:27:35,472 copying colossalai/amp/torch_amp/torch_amp.py -> build/lib/colossalai/amp/torch_amp 2023-03-10T07:27:35,475 copying colossalai/amp/torch_amp/__init__.py -> build/lib/colossalai/amp/torch_amp 2023-03-10T07:27:35,480 creating build/lib/colossalai/amp/apex_amp 2023-03-10T07:27:35,482 copying colossalai/amp/apex_amp/__init__.py -> build/lib/colossalai/amp/apex_amp 2023-03-10T07:27:35,485 copying colossalai/amp/apex_amp/apex_amp.py -> build/lib/colossalai/amp/apex_amp 2023-03-10T07:27:35,490 creating build/lib/colossalai/amp/naive_amp 2023-03-10T07:27:35,492 copying colossalai/amp/naive_amp/_utils.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T07:27:35,496 copying colossalai/amp/naive_amp/_fp16_optimizer.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T07:27:35,501 copying colossalai/amp/naive_amp/__init__.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T07:27:35,504 copying colossalai/amp/naive_amp/naive_amp.py -> build/lib/colossalai/amp/naive_amp 2023-03-10T07:27:35,510 creating build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:35,512 copying colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:35,516 copying colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:35,520 copying colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:35,523 copying colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:35,528 creating build/lib/colossalai/fx/codegen 2023-03-10T07:27:35,530 copying colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/lib/colossalai/fx/codegen 2023-03-10T07:27:35,536 copying colossalai/fx/codegen/__init__.py -> build/lib/colossalai/fx/codegen 2023-03-10T07:27:35,541 creating build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,543 copying colossalai/fx/tracer/tracer.py -> build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,549 copying colossalai/fx/tracer/registry.py -> build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,553 copying colossalai/fx/tracer/_meta_trace.py -> build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,557 copying colossalai/fx/tracer/_tracer_utils.py -> build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,561 copying colossalai/fx/tracer/experimental.py -> build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,567 copying colossalai/fx/tracer/__init__.py -> build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,570 copying colossalai/fx/tracer/_symbolic_trace.py -> build/lib/colossalai/fx/tracer 2023-03-10T07:27:35,575 creating build/lib/colossalai/fx/passes 2023-03-10T07:27:35,577 copying colossalai/fx/passes/meta_info_prop.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,582 copying colossalai/fx/passes/passes_for_gpt2_test.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,587 copying colossalai/fx/passes/adding_split_node_pass.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,591 copying colossalai/fx/passes/split_module.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,596 copying colossalai/fx/passes/shard_1d_pass.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,600 copying colossalai/fx/passes/utils.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,604 copying colossalai/fx/passes/concrete_info_prop.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,608 copying colossalai/fx/passes/__init__.py -> build/lib/colossalai/fx/passes 2023-03-10T07:27:35,613 creating build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,615 copying colossalai/fx/profiler/memory_utils.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,619 copying colossalai/fx/profiler/dataflow.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,623 copying colossalai/fx/profiler/opcount.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,628 copying colossalai/fx/profiler/__init__.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,631 copying colossalai/fx/profiler/shard_utils.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,635 copying colossalai/fx/profiler/tensor.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,639 copying colossalai/fx/profiler/constants.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,642 copying colossalai/fx/profiler/profiler.py -> build/lib/colossalai/fx/profiler 2023-03-10T07:27:35,648 creating build/lib/colossalai/fx/tracer/meta_patch 2023-03-10T07:27:35,650 copying colossalai/fx/tracer/meta_patch/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch 2023-03-10T07:27:35,655 creating build/lib/colossalai/fx/tracer/bias_addition_patch 2023-03-10T07:27:35,656 copying colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch 2023-03-10T07:27:35,662 creating build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,664 copying colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,668 copying colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,673 copying colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,677 copying colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,681 copying colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,685 copying colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,688 copying colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,692 copying colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:35,698 creating build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,700 copying colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,704 copying colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,707 copying colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,712 copying colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,716 copying colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,720 copying colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,723 copying colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,727 copying colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:35,732 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:35,734 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:35,738 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:35,742 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:35,746 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:35,751 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:35,753 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:35,757 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:35,761 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:35,764 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:35,768 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:35,773 creating build/lib/colossalai/fx/profiler/experimental 2023-03-10T07:27:35,775 copying colossalai/fx/profiler/experimental/registry.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T07:27:35,779 copying colossalai/fx/profiler/experimental/__init__.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T07:27:35,782 copying colossalai/fx/profiler/experimental/shard_utils.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T07:27:35,786 copying colossalai/fx/profiler/experimental/constants.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T07:27:35,789 copying colossalai/fx/profiler/experimental/profiler.py -> build/lib/colossalai/fx/profiler/experimental 2023-03-10T07:27:35,795 creating build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,797 copying colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,801 copying colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,805 copying colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,808 copying colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,812 copying colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,815 copying colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,819 copying colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,822 copying colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,826 copying colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,829 copying colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,833 copying colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:35,839 creating build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,841 copying colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,844 copying colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,848 copying colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,851 copying colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,855 copying colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,858 copying colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,862 copying colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,865 copying colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,869 copying colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:35,874 creating build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,876 copying colossalai/utils/checkpoint_io/writer.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,881 copying colossalai/utils/checkpoint_io/convertor.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,886 copying colossalai/utils/checkpoint_io/constant.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,889 copying colossalai/utils/checkpoint_io/utils.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,894 copying colossalai/utils/checkpoint_io/__init__.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,897 copying colossalai/utils/checkpoint_io/distributed.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,902 copying colossalai/utils/checkpoint_io/backend.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,906 copying colossalai/utils/checkpoint_io/io.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,910 copying colossalai/utils/checkpoint_io/meta.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,914 copying colossalai/utils/checkpoint_io/reader.py -> build/lib/colossalai/utils/checkpoint_io 2023-03-10T07:27:35,920 creating build/lib/colossalai/utils/model 2023-03-10T07:27:35,922 copying colossalai/utils/model/experimental.py -> build/lib/colossalai/utils/model 2023-03-10T07:27:35,928 copying colossalai/utils/model/utils.py -> build/lib/colossalai/utils/model 2023-03-10T07:27:35,932 copying colossalai/utils/model/colo_init_context.py -> build/lib/colossalai/utils/model 2023-03-10T07:27:35,935 copying colossalai/utils/model/__init__.py -> build/lib/colossalai/utils/model 2023-03-10T07:27:35,938 copying colossalai/utils/model/lazy_init_context.py -> build/lib/colossalai/utils/model 2023-03-10T07:27:35,944 creating build/lib/colossalai/utils/multi_tensor_apply 2023-03-10T07:27:35,946 copying colossalai/utils/multi_tensor_apply/__init__.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-03-10T07:27:35,950 copying colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-03-10T07:27:35,955 creating build/lib/colossalai/utils/checkpoint 2023-03-10T07:27:35,957 copying colossalai/utils/checkpoint/module_checkpoint.py -> build/lib/colossalai/utils/checkpoint 2023-03-10T07:27:35,961 copying colossalai/utils/checkpoint/utils.py -> build/lib/colossalai/utils/checkpoint 2023-03-10T07:27:35,965 copying colossalai/utils/checkpoint/__init__.py -> build/lib/colossalai/utils/checkpoint 2023-03-10T07:27:35,970 creating build/lib/colossalai/utils/data_sampler 2023-03-10T07:27:35,972 copying colossalai/utils/data_sampler/__init__.py -> build/lib/colossalai/utils/data_sampler 2023-03-10T07:27:35,976 copying colossalai/utils/data_sampler/base_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-03-10T07:27:35,979 copying colossalai/utils/data_sampler/data_parallel_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-03-10T07:27:35,985 creating build/lib/colossalai/utils/profiler 2023-03-10T07:27:35,987 copying colossalai/utils/profiler/extention.py -> build/lib/colossalai/utils/profiler 2023-03-10T07:27:35,991 copying colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/lib/colossalai/utils/profiler 2023-03-10T07:27:35,995 copying colossalai/utils/profiler/__init__.py -> build/lib/colossalai/utils/profiler 2023-03-10T07:27:35,998 copying colossalai/utils/profiler/profiler.py -> build/lib/colossalai/utils/profiler 2023-03-10T07:27:36,004 creating build/lib/colossalai/utils/tensor_detector 2023-03-10T07:27:36,006 copying colossalai/utils/tensor_detector/tensor_detector.py -> build/lib/colossalai/utils/tensor_detector 2023-03-10T07:27:36,011 copying colossalai/utils/tensor_detector/__init__.py -> build/lib/colossalai/utils/tensor_detector 2023-03-10T07:27:36,016 creating build/lib/colossalai/utils/rank_recorder 2023-03-10T07:27:36,018 copying colossalai/utils/rank_recorder/rank_recorder.py -> build/lib/colossalai/utils/rank_recorder 2023-03-10T07:27:36,022 copying colossalai/utils/rank_recorder/__init__.py -> build/lib/colossalai/utils/rank_recorder 2023-03-10T07:27:36,027 creating build/lib/colossalai/utils/profiler/legacy 2023-03-10T07:27:36,029 copying colossalai/utils/profiler/legacy/comm_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T07:27:36,034 copying colossalai/utils/profiler/legacy/__init__.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T07:27:36,038 copying colossalai/utils/profiler/legacy/pcie_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T07:27:36,042 copying colossalai/utils/profiler/legacy/prof_utils.py -> build/lib/colossalai/utils/profiler/legacy 2023-03-10T07:27:36,048 creating build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,050 copying colossalai/tensor/d_tensor/misc.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,054 copying colossalai/tensor/d_tensor/comm_spec.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,059 copying colossalai/tensor/d_tensor/sharding_spec.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,063 copying colossalai/tensor/d_tensor/utils.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,067 copying colossalai/tensor/d_tensor/__init__.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,070 copying colossalai/tensor/d_tensor/layout_converter.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,075 copying colossalai/tensor/d_tensor/d_tensor.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,080 copying colossalai/tensor/d_tensor/layout.py -> build/lib/colossalai/tensor/d_tensor 2023-03-10T07:27:36,085 creating build/lib/colossalai/kernel/jit 2023-03-10T07:27:36,087 copying colossalai/kernel/jit/bias_gelu.py -> build/lib/colossalai/kernel/jit 2023-03-10T07:27:36,091 copying colossalai/kernel/jit/__init__.py -> build/lib/colossalai/kernel/jit 2023-03-10T07:27:36,094 copying colossalai/kernel/jit/bias_dropout_add.py -> build/lib/colossalai/kernel/jit 2023-03-10T07:27:36,098 copying colossalai/kernel/jit/option.py -> build/lib/colossalai/kernel/jit 2023-03-10T07:27:36,104 creating build/lib/colossalai/kernel/cuda_native 2023-03-10T07:27:36,106 copying colossalai/kernel/cuda_native/scaled_softmax.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T07:27:36,111 copying colossalai/kernel/cuda_native/multihead_attention.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T07:27:36,115 copying colossalai/kernel/cuda_native/__init__.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T07:27:36,119 copying colossalai/kernel/cuda_native/flash_attention.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T07:27:36,124 copying colossalai/kernel/cuda_native/layer_norm.py -> build/lib/colossalai/kernel/cuda_native 2023-03-10T07:27:36,130 creating build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,132 copying colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,136 copying colossalai/kernel/op_builder/fused_optim.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,140 copying colossalai/kernel/op_builder/cpu_adam.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,144 copying colossalai/kernel/op_builder/multi_head_attn.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,148 copying colossalai/kernel/op_builder/moe.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,152 copying colossalai/kernel/op_builder/utils.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,156 copying colossalai/kernel/op_builder/__init__.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,159 copying colossalai/kernel/op_builder/builder.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,164 copying colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,168 copying colossalai/kernel/op_builder/layernorm.py -> build/lib/colossalai/kernel/op_builder 2023-03-10T07:27:36,173 creating build/lib/colossalai/pipeline/rpc 2023-03-10T07:27:36,175 copying colossalai/pipeline/rpc/_pipeline_base.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T07:27:36,183 copying colossalai/pipeline/rpc/_pipeline_schedule.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T07:27:36,188 copying colossalai/pipeline/rpc/utils.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T07:27:36,193 copying colossalai/pipeline/rpc/__init__.py -> build/lib/colossalai/pipeline/rpc 2023-03-10T07:27:36,198 creating build/lib/colossalai/pipeline/middleware 2023-03-10T07:27:36,200 copying colossalai/pipeline/middleware/topo.py -> build/lib/colossalai/pipeline/middleware 2023-03-10T07:27:36,204 copying colossalai/pipeline/middleware/__init__.py -> build/lib/colossalai/pipeline/middleware 2023-03-10T07:27:36,209 creating build/lib/colossalai/pipeline/middleware/adaptor 2023-03-10T07:27:36,210 copying colossalai/pipeline/middleware/adaptor/__init__.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-03-10T07:27:36,214 copying colossalai/pipeline/middleware/adaptor/fx.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-03-10T07:27:36,221 creating build/lib/colossalai/engine/gradient_accumulation 2023-03-10T07:27:36,222 copying colossalai/engine/gradient_accumulation/__init__.py -> build/lib/colossalai/engine/gradient_accumulation 2023-03-10T07:27:36,226 copying colossalai/engine/gradient_accumulation/_gradient_accumulation.py -> build/lib/colossalai/engine/gradient_accumulation 2023-03-10T07:27:36,232 creating build/lib/colossalai/engine/schedule 2023-03-10T07:27:36,233 copying colossalai/engine/schedule/_pipeline_schedule_v2.py -> build/lib/colossalai/engine/schedule 2023-03-10T07:27:36,238 copying colossalai/engine/schedule/_base_schedule.py -> build/lib/colossalai/engine/schedule 2023-03-10T07:27:36,242 copying colossalai/engine/schedule/_non_pipeline_schedule.py -> build/lib/colossalai/engine/schedule 2023-03-10T07:27:36,245 copying colossalai/engine/schedule/_pipeline_schedule.py -> build/lib/colossalai/engine/schedule 2023-03-10T07:27:36,252 copying colossalai/engine/schedule/__init__.py -> build/lib/colossalai/engine/schedule 2023-03-10T07:27:36,257 creating build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,259 copying colossalai/engine/gradient_handler/_moe_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,262 copying colossalai/engine/gradient_handler/_zero_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,266 copying colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,269 copying colossalai/engine/gradient_handler/_base_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,273 copying colossalai/engine/gradient_handler/utils.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,276 copying colossalai/engine/gradient_handler/__init__.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,280 copying colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,283 copying colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-03-10T07:27:36,288 creating build/lib/colossalai/context/random 2023-03-10T07:27:36,290 copying colossalai/context/random/seed_manager.py -> build/lib/colossalai/context/random 2023-03-10T07:27:36,294 copying colossalai/context/random/__init__.py -> build/lib/colossalai/context/random 2023-03-10T07:27:36,297 copying colossalai/context/random/_helper.py -> build/lib/colossalai/context/random 2023-03-10T07:27:36,304 creating build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,305 copying colossalai/context/process_group_initializer/initializer_data.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,309 copying colossalai/context/process_group_initializer/initializer_2p5d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,314 copying colossalai/context/process_group_initializer/initializer_pipeline.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,317 copying colossalai/context/process_group_initializer/initializer_2d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,321 copying colossalai/context/process_group_initializer/initializer_3d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,325 copying colossalai/context/process_group_initializer/__init__.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,329 copying colossalai/context/process_group_initializer/initializer_model.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,333 copying colossalai/context/process_group_initializer/initializer_sequence.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,336 copying colossalai/context/process_group_initializer/process_group_initializer.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,340 copying colossalai/context/process_group_initializer/initializer_tensor.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,344 copying colossalai/context/process_group_initializer/initializer_1d.py -> build/lib/colossalai/context/process_group_initializer 2023-03-10T07:27:36,349 creating build/lib/colossalai/zero/init_ctx 2023-03-10T07:27:36,350 copying colossalai/zero/init_ctx/init_context.py -> build/lib/colossalai/zero/init_ctx 2023-03-10T07:27:36,355 copying colossalai/zero/init_ctx/__init__.py -> build/lib/colossalai/zero/init_ctx 2023-03-10T07:27:36,360 creating build/lib/colossalai/zero/shard_utils 2023-03-10T07:27:36,362 copying colossalai/zero/shard_utils/base_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T07:27:36,366 copying colossalai/zero/shard_utils/commons.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T07:27:36,370 copying colossalai/zero/shard_utils/tensor_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T07:27:36,374 copying colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T07:27:36,377 copying colossalai/zero/shard_utils/__init__.py -> build/lib/colossalai/zero/shard_utils 2023-03-10T07:27:36,382 creating build/lib/colossalai/zero/sharded_model 2023-03-10T07:27:36,384 copying colossalai/zero/sharded_model/sharded_model_v2.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T07:27:36,391 copying colossalai/zero/sharded_model/_utils.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T07:27:36,395 copying colossalai/zero/sharded_model/reduce_scatter.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T07:27:36,399 copying colossalai/zero/sharded_model/utils.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T07:27:36,403 copying colossalai/zero/sharded_model/__init__.py -> build/lib/colossalai/zero/sharded_model 2023-03-10T07:27:36,407 creating build/lib/colossalai/zero/utils 2023-03-10T07:27:36,409 copying colossalai/zero/utils/gemini_hook.py -> build/lib/colossalai/zero/utils 2023-03-10T07:27:36,413 copying colossalai/zero/utils/__init__.py -> build/lib/colossalai/zero/utils 2023-03-10T07:27:36,417 copying colossalai/zero/utils/zero_hook.py -> build/lib/colossalai/zero/utils 2023-03-10T07:27:36,423 creating build/lib/colossalai/zero/sharded_param 2023-03-10T07:27:36,425 copying colossalai/zero/sharded_param/sharded_tensor.py -> build/lib/colossalai/zero/sharded_param 2023-03-10T07:27:36,428 copying colossalai/zero/sharded_param/__init__.py -> build/lib/colossalai/zero/sharded_param 2023-03-10T07:27:36,432 copying colossalai/zero/sharded_param/sharded_param.py -> build/lib/colossalai/zero/sharded_param 2023-03-10T07:27:36,437 creating build/lib/colossalai/zero/sharded_optim 2023-03-10T07:27:36,439 copying colossalai/zero/sharded_optim/_utils.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T07:27:36,444 copying colossalai/zero/sharded_optim/sharded_optim_v2.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T07:27:36,450 copying colossalai/zero/sharded_optim/low_level_optim.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T07:27:36,455 copying colossalai/zero/sharded_optim/__init__.py -> build/lib/colossalai/zero/sharded_optim 2023-03-10T07:27:36,461 creating build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:36,463 copying colossalai/zero/sharded_optim/bookkeeping/base_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:36,468 copying colossalai/zero/sharded_optim/bookkeeping/bucket_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:36,472 copying colossalai/zero/sharded_optim/bookkeeping/parameter_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:36,476 copying colossalai/zero/sharded_optim/bookkeeping/__init__.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:36,481 copying colossalai/zero/sharded_optim/bookkeeping/gradient_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:36,485 copying colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:36,514 creating build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,516 copying colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,522 copying colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,528 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,532 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,539 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,545 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,550 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,556 copying colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,561 copying colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,566 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,572 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,577 copying colossalai/kernel/cuda_native/csrc/type_shim.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,583 copying colossalai/kernel/cuda_native/csrc/compat.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,588 copying colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,593 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,600 copying colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,606 copying colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,612 copying colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,617 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,622 copying colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,628 copying colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,633 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,638 copying colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:36,642 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels 2023-03-10T07:27:36,643 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,645 copying colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,651 copying colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,655 copying colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,659 copying colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,664 copying colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,668 copying colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,672 copying colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,676 copying colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,681 copying colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,686 copying colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,690 copying colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,694 copying colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:36,920 /usr/local/lib/python3.7/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. 2023-03-10T07:27:36,920 setuptools.SetuptoolsDeprecationWarning, 2023-03-10T07:27:36,979 installing to build/bdist.linux-armv7l/wheel 2023-03-10T07:27:36,979 running install 2023-03-10T07:27:37,039 running install_lib 2023-03-10T07:27:37,102 creating build/bdist.linux-armv7l 2023-03-10T07:27:37,102 creating build/bdist.linux-armv7l/wheel 2023-03-10T07:27:37,107 creating build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T07:27:37,110 creating build/bdist.linux-armv7l/wheel/colossalai/logging 2023-03-10T07:27:37,112 copying build/lib/colossalai/logging/logger.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-03-10T07:27:37,117 copying build/lib/colossalai/logging/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-03-10T07:27:37,122 creating build/bdist.linux-armv7l/wheel/colossalai/cli 2023-03-10T07:27:37,125 creating build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T07:27:37,128 copying build/lib/colossalai/cli/launcher/run.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T07:27:37,133 copying build/lib/colossalai/cli/launcher/multinode_runner.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T07:27:37,137 copying build/lib/colossalai/cli/launcher/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T07:27:37,141 copying build/lib/colossalai/cli/launcher/hostinfo.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-03-10T07:27:37,147 creating build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-03-10T07:27:37,148 copying build/lib/colossalai/cli/check/check_installation.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-03-10T07:27:37,154 copying build/lib/colossalai/cli/check/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-03-10T07:27:37,157 copying build/lib/colossalai/cli/cli.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-03-10T07:27:37,161 copying build/lib/colossalai/cli/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-03-10T07:27:37,166 creating build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T07:27:37,168 copying build/lib/colossalai/cli/benchmark/benchmark.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T07:27:37,173 copying build/lib/colossalai/cli/benchmark/models.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T07:27:37,177 copying build/lib/colossalai/cli/benchmark/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T07:27:37,181 copying build/lib/colossalai/cli/benchmark/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-03-10T07:27:37,186 creating build/bdist.linux-armv7l/wheel/colossalai/builder 2023-03-10T07:27:37,187 copying build/lib/colossalai/builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/builder 2023-03-10T07:27:37,191 copying build/lib/colossalai/builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/builder 2023-03-10T07:27:37,196 creating build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T07:27:37,198 copying build/lib/colossalai/booster/precision.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T07:27:37,202 copying build/lib/colossalai/booster/plugin.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T07:27:37,205 copying build/lib/colossalai/booster/environment_table.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T07:27:37,209 copying build/lib/colossalai/booster/booster.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T07:27:37,212 copying build/lib/colossalai/booster/accelerator.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T07:27:37,216 copying build/lib/colossalai/booster/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/booster 2023-03-10T07:27:37,220 copying build/lib/colossalai/global_variables.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T07:27:37,225 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-03-10T07:27:37,228 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:37,230 copying build/lib/colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:37,236 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:37,238 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:37,242 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:37,246 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:37,251 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:37,255 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:37,259 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-03-10T07:27:37,263 copying build/lib/colossalai/auto_parallel/tensor_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:37,266 copying build/lib/colossalai/auto_parallel/tensor_shard/options.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:37,271 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,273 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,277 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,281 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,284 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,288 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,292 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,298 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,304 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,307 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,311 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,314 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,318 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,324 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,328 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,331 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,336 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,341 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,344 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,349 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,352 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,358 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,360 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,364 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,368 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,373 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,378 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,382 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,386 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,390 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,395 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,399 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,402 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,408 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,412 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,417 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,423 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,427 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,430 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,435 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,439 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-03-10T07:27:37,445 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,449 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,453 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,457 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,460 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,465 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,469 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-03-10T07:27:37,473 copying build/lib/colossalai/auto_parallel/tensor_shard/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:37,479 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:37,481 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:37,486 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:37,491 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:37,495 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:37,500 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-03-10T07:27:37,505 copying build/lib/colossalai/auto_parallel/tensor_shard/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-03-10T07:27:37,512 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:37,514 copying build/lib/colossalai/auto_parallel/meta_profiler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:37,519 copying build/lib/colossalai/auto_parallel/meta_profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:37,524 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,526 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/where.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,531 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,537 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,541 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,546 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,550 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,555 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,559 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,563 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,568 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,572 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-03-10T07:27:37,576 copying build/lib/colossalai/auto_parallel/meta_profiler/metainfo.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:37,580 copying build/lib/colossalai/auto_parallel/meta_profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-03-10T07:27:37,584 copying build/lib/colossalai/auto_parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-03-10T07:27:37,589 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:37,591 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:37,597 copying build/lib/colossalai/auto_parallel/checkpoint/operation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:37,602 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:37,607 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:37,611 copying build/lib/colossalai/auto_parallel/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:37,616 copying build/lib/colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-03-10T07:27:37,621 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T07:27:37,623 copying build/lib/colossalai/auto_parallel/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T07:27:37,629 copying build/lib/colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T07:27:37,634 copying build/lib/colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T07:27:37,638 copying build/lib/colossalai/auto_parallel/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T07:27:37,642 copying build/lib/colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T07:27:37,648 copying build/lib/colossalai/auto_parallel/passes/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-03-10T07:27:37,654 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-03-10T07:27:37,656 copying build/lib/colossalai/auto_parallel/pipeline_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-03-10T07:27:37,661 creating build/bdist.linux-armv7l/wheel/colossalai/nn 2023-03-10T07:27:37,664 creating build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,667 copying build/lib/colossalai/nn/_ops/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,672 copying build/lib/colossalai/nn/_ops/element_wise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,677 copying build/lib/colossalai/nn/_ops/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,681 copying build/lib/colossalai/nn/_ops/embedding_bag.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,686 copying build/lib/colossalai/nn/_ops/view.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,689 copying build/lib/colossalai/nn/_ops/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,693 copying build/lib/colossalai/nn/_ops/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,697 copying build/lib/colossalai/nn/_ops/loss.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,701 copying build/lib/colossalai/nn/_ops/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,704 copying build/lib/colossalai/nn/_ops/batch_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,708 copying build/lib/colossalai/nn/_ops/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-03-10T07:27:37,712 creating build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,714 copying build/lib/colossalai/nn/lr_scheduler/onecycle.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,719 copying build/lib/colossalai/nn/lr_scheduler/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,722 copying build/lib/colossalai/nn/lr_scheduler/torch.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,726 copying build/lib/colossalai/nn/lr_scheduler/poly.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,730 copying build/lib/colossalai/nn/lr_scheduler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,734 copying build/lib/colossalai/nn/lr_scheduler/multistep.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,737 copying build/lib/colossalai/nn/lr_scheduler/cosine.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,741 copying build/lib/colossalai/nn/lr_scheduler/delayed.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-03-10T07:27:37,747 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T07:27:37,749 copying build/lib/colossalai/nn/parallel/reducer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T07:27:37,753 copying build/lib/colossalai/nn/parallel/zero_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T07:27:37,757 copying build/lib/colossalai/nn/parallel/data_parallel.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T07:27:37,763 copying build/lib/colossalai/nn/parallel/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T07:27:37,767 copying build/lib/colossalai/nn/parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T07:27:37,772 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T07:27:37,775 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,777 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,781 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,786 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,791 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,795 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,799 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,803 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,807 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,811 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-03-10T07:27:37,815 copying build/lib/colossalai/nn/parallel/layers/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T07:27:37,819 copying build/lib/colossalai/nn/parallel/layers/colo_module.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T07:27:37,822 copying build/lib/colossalai/nn/parallel/layers/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T07:27:37,826 copying build/lib/colossalai/nn/parallel/layers/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T07:27:37,830 copying build/lib/colossalai/nn/parallel/layers/module_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-03-10T07:27:37,834 copying build/lib/colossalai/nn/parallel/gemini_parallel.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-03-10T07:27:37,839 creating build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T07:27:37,840 copying build/lib/colossalai/nn/loss/loss_moe.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T07:27:37,844 copying build/lib/colossalai/nn/loss/loss_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T07:27:37,848 copying build/lib/colossalai/nn/loss/loss_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T07:27:37,852 copying build/lib/colossalai/nn/loss/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T07:27:37,856 copying build/lib/colossalai/nn/loss/loss_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T07:27:37,860 copying build/lib/colossalai/nn/loss/loss_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-03-10T07:27:37,866 creating build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,867 copying build/lib/colossalai/nn/optimizer/lars.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,871 copying build/lib/colossalai/nn/optimizer/fused_sgd.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,876 copying build/lib/colossalai/nn/optimizer/gemini_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,881 copying build/lib/colossalai/nn/optimizer/lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,885 copying build/lib/colossalai/nn/optimizer/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,889 copying build/lib/colossalai/nn/optimizer/colossalai_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,893 copying build/lib/colossalai/nn/optimizer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,896 copying build/lib/colossalai/nn/optimizer/zero_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,901 copying build/lib/colossalai/nn/optimizer/fused_lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,906 copying build/lib/colossalai/nn/optimizer/hybrid_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,910 copying build/lib/colossalai/nn/optimizer/nvme_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,914 copying build/lib/colossalai/nn/optimizer/fused_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-03-10T07:27:37,919 copying build/lib/colossalai/nn/init.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-03-10T07:27:37,923 copying build/lib/colossalai/nn/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-03-10T07:27:37,928 creating build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T07:27:37,930 copying build/lib/colossalai/nn/metric/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T07:27:37,933 copying build/lib/colossalai/nn/metric/accuracy_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T07:27:37,937 copying build/lib/colossalai/nn/metric/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T07:27:37,941 copying build/lib/colossalai/nn/metric/accuracy_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T07:27:37,944 copying build/lib/colossalai/nn/metric/accuracy_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-03-10T07:27:37,949 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-03-10T07:27:37,952 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T07:27:37,954 copying build/lib/colossalai/nn/layer/moe/routers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T07:27:37,958 copying build/lib/colossalai/nn/layer/moe/experts.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T07:27:37,963 copying build/lib/colossalai/nn/layer/moe/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T07:27:37,967 copying build/lib/colossalai/nn/layer/moe/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T07:27:37,971 copying build/lib/colossalai/nn/layer/moe/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T07:27:37,975 copying build/lib/colossalai/nn/layer/moe/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-03-10T07:27:37,979 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:37,981 copying build/lib/colossalai/nn/layer/parallel_2d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:37,985 copying build/lib/colossalai/nn/layer/parallel_2d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:37,992 copying build/lib/colossalai/nn/layer/parallel_2d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:37,998 copying build/lib/colossalai/nn/layer/parallel_2d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-03-10T07:27:38,002 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-03-10T07:27:38,004 copying build/lib/colossalai/nn/layer/vanilla/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-03-10T07:27:38,009 copying build/lib/colossalai/nn/layer/vanilla/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-03-10T07:27:38,013 copying build/lib/colossalai/nn/layer/base_layer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-03-10T07:27:38,017 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:38,019 copying build/lib/colossalai/nn/layer/parallel_2p5d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:38,023 copying build/lib/colossalai/nn/layer/parallel_2p5d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:38,029 copying build/lib/colossalai/nn/layer/parallel_2p5d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:38,035 copying build/lib/colossalai/nn/layer/parallel_2p5d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-03-10T07:27:38,040 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:38,042 copying build/lib/colossalai/nn/layer/parallel_sequence/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:38,046 copying build/lib/colossalai/nn/layer/parallel_sequence/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:38,050 copying build/lib/colossalai/nn/layer/parallel_sequence/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:38,054 copying build/lib/colossalai/nn/layer/parallel_sequence/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-03-10T07:27:38,059 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-03-10T07:27:38,061 copying build/lib/colossalai/nn/layer/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-03-10T07:27:38,065 copying build/lib/colossalai/nn/layer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-03-10T07:27:38,068 copying build/lib/colossalai/nn/layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-03-10T07:27:38,073 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:38,075 copying build/lib/colossalai/nn/layer/parallel_1d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:38,079 copying build/lib/colossalai/nn/layer/parallel_1d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:38,085 copying build/lib/colossalai/nn/layer/parallel_1d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:38,089 copying build/lib/colossalai/nn/layer/parallel_1d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-03-10T07:27:38,094 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:38,096 copying build/lib/colossalai/nn/layer/colossalai_layer/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:38,099 copying build/lib/colossalai/nn/layer/colossalai_layer/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:38,103 copying build/lib/colossalai/nn/layer/colossalai_layer/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:38,107 copying build/lib/colossalai/nn/layer/colossalai_layer/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:38,111 copying build/lib/colossalai/nn/layer/colossalai_layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:38,115 copying build/lib/colossalai/nn/layer/colossalai_layer/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-03-10T07:27:38,120 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-03-10T07:27:38,122 copying build/lib/colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-03-10T07:27:38,126 copying build/lib/colossalai/nn/layer/wrapper/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-03-10T07:27:38,131 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:38,133 copying build/lib/colossalai/nn/layer/parallel_3d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:38,137 copying build/lib/colossalai/nn/layer/parallel_3d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:38,143 copying build/lib/colossalai/nn/layer/parallel_3d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:38,149 copying build/lib/colossalai/nn/layer/parallel_3d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-03-10T07:27:38,154 creating build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T07:27:38,155 copying build/lib/colossalai/device/alpha_beta_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T07:27:38,160 copying build/lib/colossalai/device/calc_pipeline_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T07:27:38,165 copying build/lib/colossalai/device/device_mesh.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T07:27:38,170 copying build/lib/colossalai/device/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-03-10T07:27:38,175 creating build/bdist.linux-armv7l/wheel/colossalai/registry 2023-03-10T07:27:38,177 copying build/lib/colossalai/registry/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/registry 2023-03-10T07:27:38,182 copying build/lib/colossalai/registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/registry 2023-03-10T07:27:38,188 creating build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-03-10T07:27:38,190 copying build/lib/colossalai/trainer/_trainer.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-03-10T07:27:38,195 copying build/lib/colossalai/trainer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-03-10T07:27:38,201 creating build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,203 copying build/lib/colossalai/trainer/hooks/_checkpoint_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,208 copying build/lib/colossalai/trainer/hooks/_lr_scheduler_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,212 copying build/lib/colossalai/trainer/hooks/_commons_.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,216 copying build/lib/colossalai/trainer/hooks/_metric_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,221 copying build/lib/colossalai/trainer/hooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,226 copying build/lib/colossalai/trainer/hooks/_base_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,230 copying build/lib/colossalai/trainer/hooks/_log_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-03-10T07:27:38,237 creating build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T07:27:38,239 copying build/lib/colossalai/testing/pytest_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T07:27:38,243 copying build/lib/colossalai/testing/random.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T07:27:38,247 copying build/lib/colossalai/testing/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T07:27:38,252 copying build/lib/colossalai/testing/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T07:27:38,256 copying build/lib/colossalai/testing/comparison.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-03-10T07:27:38,262 creating build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,264 copying build/lib/colossalai/gemini/stateful_tensor_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,269 copying build/lib/colossalai/gemini/gemini_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,274 copying build/lib/colossalai/gemini/placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,279 copying build/lib/colossalai/gemini/tensor_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,284 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T07:27:38,285 copying build/lib/colossalai/gemini/chunk/manager.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T07:27:38,290 copying build/lib/colossalai/gemini/chunk/search_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T07:27:38,294 copying build/lib/colossalai/gemini/chunk/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T07:27:38,298 copying build/lib/colossalai/gemini/chunk/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T07:27:38,301 copying build/lib/colossalai/gemini/chunk/chunk.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-03-10T07:27:38,307 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,309 copying build/lib/colossalai/gemini/memory_tracer/static_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,314 copying build/lib/colossalai/gemini/memory_tracer/chunk_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,318 copying build/lib/colossalai/gemini/memory_tracer/param_runtime_order.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,322 copying build/lib/colossalai/gemini/memory_tracer/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,326 copying build/lib/colossalai/gemini/memory_tracer/memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,330 copying build/lib/colossalai/gemini/memory_tracer/memory_monitor.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,335 copying build/lib/colossalai/gemini/memory_tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,339 copying build/lib/colossalai/gemini/memory_tracer/runtime_mem_tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,343 copying build/lib/colossalai/gemini/memory_tracer/memory_stats.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-03-10T07:27:38,348 copying build/lib/colossalai/gemini/stateful_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,352 copying build/lib/colossalai/gemini/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,356 copying build/lib/colossalai/gemini/gemini_context.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,361 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-03-10T07:27:38,364 copying build/lib/colossalai/gemini/paramhooks/_param_hookmgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-03-10T07:27:38,368 copying build/lib/colossalai/gemini/paramhooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-03-10T07:27:38,373 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T07:27:38,376 copying build/lib/colossalai/gemini/ophooks/_shard_param_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T07:27:38,380 copying build/lib/colossalai/gemini/ophooks/_shard_grad_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T07:27:38,384 copying build/lib/colossalai/gemini/ophooks/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T07:27:38,628 copying build/lib/colossalai/gemini/ophooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T07:27:38,632 copying build/lib/colossalai/gemini/ophooks/runtime_mem_tracer_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-03-10T07:27:38,637 copying build/lib/colossalai/gemini/tensor_placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-03-10T07:27:38,643 creating build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T07:27:38,646 copying build/lib/colossalai/communication/ring.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T07:27:38,650 copying build/lib/colossalai/communication/collective.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T07:27:38,655 copying build/lib/colossalai/communication/p2p.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T07:27:38,661 copying build/lib/colossalai/communication/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T07:27:38,666 copying build/lib/colossalai/communication/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T07:27:38,670 copying build/lib/colossalai/communication/p2p_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-03-10T07:27:38,677 creating build/bdist.linux-armv7l/wheel/colossalai/amp 2023-03-10T07:27:38,679 creating build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T07:27:38,681 copying build/lib/colossalai/amp/torch_amp/_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T07:27:38,687 copying build/lib/colossalai/amp/torch_amp/torch_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T07:27:38,690 copying build/lib/colossalai/amp/torch_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-03-10T07:27:38,695 creating build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-03-10T07:27:38,697 copying build/lib/colossalai/amp/apex_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-03-10T07:27:38,701 copying build/lib/colossalai/amp/apex_amp/apex_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-03-10T07:27:38,705 copying build/lib/colossalai/amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-03-10T07:27:38,709 copying build/lib/colossalai/amp/amp_type.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-03-10T07:27:38,713 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T07:27:38,715 copying build/lib/colossalai/amp/naive_amp/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T07:27:38,720 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:38,722 copying build/lib/colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:38,727 copying build/lib/colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:38,731 copying build/lib/colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:38,734 copying build/lib/colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-03-10T07:27:38,738 copying build/lib/colossalai/amp/naive_amp/_fp16_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T07:27:38,743 copying build/lib/colossalai/amp/naive_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T07:27:38,746 copying build/lib/colossalai/amp/naive_amp/naive_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-03-10T07:27:38,751 copying build/lib/colossalai/version.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T07:27:38,755 creating build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T07:27:38,758 creating build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-03-10T07:27:38,760 copying build/lib/colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-03-10T07:27:38,766 copying build/lib/colossalai/fx/codegen/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-03-10T07:27:38,772 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,774 copying build/lib/colossalai/fx/tracer/tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,780 copying build/lib/colossalai/fx/tracer/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,784 copying build/lib/colossalai/fx/tracer/_meta_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,789 copying build/lib/colossalai/fx/tracer/_tracer_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,795 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-03-10T07:27:38,798 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,801 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,806 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,811 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,816 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,820 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,825 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,829 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,833 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-03-10T07:27:38,838 copying build/lib/colossalai/fx/tracer/meta_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-03-10T07:27:38,843 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,846 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,851 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,855 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,860 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,865 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,870 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,874 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,878 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-03-10T07:27:38,883 copying build/lib/colossalai/fx/tracer/experimental.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,890 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-03-10T07:27:38,893 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:38,896 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:38,900 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:38,905 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:38,910 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-03-10T07:27:38,915 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:38,918 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:38,922 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:38,927 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:38,932 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:38,936 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-03-10T07:27:38,941 copying build/lib/colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-03-10T07:27:38,945 copying build/lib/colossalai/fx/tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,949 copying build/lib/colossalai/fx/tracer/_symbolic_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-03-10T07:27:38,953 copying build/lib/colossalai/fx/_compatibility.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T07:27:38,957 copying build/lib/colossalai/fx/_meta_registrations.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T07:27:38,962 copying build/lib/colossalai/fx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T07:27:38,967 creating build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:38,969 copying build/lib/colossalai/fx/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:38,973 copying build/lib/colossalai/fx/passes/passes_for_gpt2_test.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:38,978 copying build/lib/colossalai/fx/passes/adding_split_node_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:38,983 copying build/lib/colossalai/fx/passes/split_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:38,988 copying build/lib/colossalai/fx/passes/shard_1d_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:38,992 copying build/lib/colossalai/fx/passes/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:38,996 copying build/lib/colossalai/fx/passes/concrete_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:39,001 copying build/lib/colossalai/fx/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-03-10T07:27:39,006 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,008 copying build/lib/colossalai/fx/profiler/memory_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,011 copying build/lib/colossalai/fx/profiler/dataflow.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,015 copying build/lib/colossalai/fx/profiler/opcount.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,020 copying build/lib/colossalai/fx/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,023 copying build/lib/colossalai/fx/profiler/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,028 copying build/lib/colossalai/fx/profiler/tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,032 copying build/lib/colossalai/fx/profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,036 copying build/lib/colossalai/fx/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-03-10T07:27:39,041 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T07:27:39,044 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,046 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,050 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,054 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,058 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,062 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,065 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,069 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,073 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,076 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,080 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,083 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-03-10T07:27:39,087 copying build/lib/colossalai/fx/profiler/experimental/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T07:27:39,092 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,094 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,098 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,102 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,105 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,109 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,113 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,116 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,120 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,123 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-03-10T07:27:39,127 copying build/lib/colossalai/fx/profiler/experimental/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T07:27:39,130 copying build/lib/colossalai/fx/profiler/experimental/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T07:27:39,134 copying build/lib/colossalai/fx/profiler/experimental/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T07:27:39,137 copying build/lib/colossalai/fx/profiler/experimental/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-03-10T07:27:39,142 copying build/lib/colossalai/fx/proxy.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T07:27:39,145 copying build/lib/colossalai/fx/graph_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-03-10T07:27:39,151 creating build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,153 copying build/lib/colossalai/utils/cuda.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,158 creating build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,159 copying build/lib/colossalai/utils/checkpoint_io/writer.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,164 copying build/lib/colossalai/utils/checkpoint_io/convertor.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,169 copying build/lib/colossalai/utils/checkpoint_io/constant.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,173 copying build/lib/colossalai/utils/checkpoint_io/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,178 copying build/lib/colossalai/utils/checkpoint_io/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,183 copying build/lib/colossalai/utils/checkpoint_io/distributed.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,187 copying build/lib/colossalai/utils/checkpoint_io/backend.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,192 copying build/lib/colossalai/utils/checkpoint_io/io.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,196 copying build/lib/colossalai/utils/checkpoint_io/meta.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,200 copying build/lib/colossalai/utils/checkpoint_io/reader.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-03-10T07:27:39,205 creating build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T07:27:39,207 copying build/lib/colossalai/utils/model/experimental.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T07:27:39,212 copying build/lib/colossalai/utils/model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T07:27:39,216 copying build/lib/colossalai/utils/model/colo_init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T07:27:39,221 copying build/lib/colossalai/utils/model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T07:27:39,224 copying build/lib/colossalai/utils/model/lazy_init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-03-10T07:27:39,229 copying build/lib/colossalai/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,235 copying build/lib/colossalai/utils/checkpointing.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,241 creating build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-03-10T07:27:39,243 copying build/lib/colossalai/utils/multi_tensor_apply/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-03-10T07:27:39,248 copying build/lib/colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-03-10T07:27:39,252 copying build/lib/colossalai/utils/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,257 copying build/lib/colossalai/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,262 creating build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T07:27:39,264 copying build/lib/colossalai/utils/checkpoint/module_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T07:27:39,270 copying build/lib/colossalai/utils/checkpoint/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T07:27:39,274 copying build/lib/colossalai/utils/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-03-10T07:27:39,279 creating build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T07:27:39,282 copying build/lib/colossalai/utils/data_sampler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T07:27:39,286 copying build/lib/colossalai/utils/data_sampler/base_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T07:27:39,290 copying build/lib/colossalai/utils/data_sampler/data_parallel_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-03-10T07:27:39,295 copying build/lib/colossalai/utils/memory.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,300 copying build/lib/colossalai/utils/activation_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,306 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T07:27:39,309 copying build/lib/colossalai/utils/profiler/extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T07:27:39,313 copying build/lib/colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T07:27:39,319 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T07:27:39,321 copying build/lib/colossalai/utils/profiler/legacy/comm_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T07:27:39,326 copying build/lib/colossalai/utils/profiler/legacy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T07:27:39,331 copying build/lib/colossalai/utils/profiler/legacy/pcie_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T07:27:39,336 copying build/lib/colossalai/utils/profiler/legacy/prof_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-03-10T07:27:39,340 copying build/lib/colossalai/utils/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T07:27:39,343 copying build/lib/colossalai/utils/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-03-10T07:27:39,349 creating build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-03-10T07:27:39,350 copying build/lib/colossalai/utils/tensor_detector/tensor_detector.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-03-10T07:27:39,355 copying build/lib/colossalai/utils/tensor_detector/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-03-10T07:27:39,360 creating build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-03-10T07:27:39,361 copying build/lib/colossalai/utils/rank_recorder/rank_recorder.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-03-10T07:27:39,366 copying build/lib/colossalai/utils/rank_recorder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-03-10T07:27:39,370 copying build/lib/colossalai/utils/timer.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-03-10T07:27:39,376 creating build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,377 copying build/lib/colossalai/tensor/param_op_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,382 copying build/lib/colossalai/tensor/shape_consistency.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,388 copying build/lib/colossalai/tensor/comm_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,393 copying build/lib/colossalai/tensor/colo_parameter.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,398 creating build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,400 copying build/lib/colossalai/tensor/d_tensor/misc.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,404 copying build/lib/colossalai/tensor/d_tensor/comm_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,408 copying build/lib/colossalai/tensor/d_tensor/sharding_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,413 copying build/lib/colossalai/tensor/d_tensor/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,416 copying build/lib/colossalai/tensor/d_tensor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,420 copying build/lib/colossalai/tensor/d_tensor/layout_converter.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,425 copying build/lib/colossalai/tensor/d_tensor/d_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,429 copying build/lib/colossalai/tensor/d_tensor/layout.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor/d_tensor 2023-03-10T07:27:39,433 copying build/lib/colossalai/tensor/sharding_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,438 copying build/lib/colossalai/tensor/compute_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,441 copying build/lib/colossalai/tensor/const.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,445 copying build/lib/colossalai/tensor/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,449 copying build/lib/colossalai/tensor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,453 copying build/lib/colossalai/tensor/colo_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,457 copying build/lib/colossalai/tensor/tensor_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,461 copying build/lib/colossalai/tensor/process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,465 copying build/lib/colossalai/tensor/op_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,469 copying build/lib/colossalai/tensor/distspec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,472 copying build/lib/colossalai/tensor/dist_spec_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-03-10T07:27:39,477 copying build/lib/colossalai/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T07:27:39,481 creating build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-03-10T07:27:39,484 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T07:27:39,486 copying build/lib/colossalai/kernel/jit/bias_gelu.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T07:27:39,490 copying build/lib/colossalai/kernel/jit/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T07:27:39,494 copying build/lib/colossalai/kernel/jit/bias_dropout_add.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T07:27:39,497 copying build/lib/colossalai/kernel/jit/option.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-03-10T07:27:39,502 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T07:27:39,503 copying build/lib/colossalai/kernel/cuda_native/scaled_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T07:27:39,510 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,511 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,516 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,521 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,524 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,530 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,534 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,539 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,544 copying build/lib/colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,548 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,552 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,558 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,561 copying build/lib/colossalai/kernel/cuda_native/csrc/type_shim.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,566 copying build/lib/colossalai/kernel/cuda_native/csrc/compat.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,569 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,574 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,577 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,582 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,586 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,591 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,594 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,600 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,605 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels 2023-03-10T07:27:39,608 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,610 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,615 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,618 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,622 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,626 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,629 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,633 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,637 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,640 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,645 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,649 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,652 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-03-10T07:27:39,656 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,660 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-03-10T07:27:39,664 copying build/lib/colossalai/kernel/cuda_native/multihead_attention.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T07:27:39,669 copying build/lib/colossalai/kernel/cuda_native/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T07:27:39,672 copying build/lib/colossalai/kernel/cuda_native/flash_attention.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T07:27:39,677 copying build/lib/colossalai/kernel/cuda_native/layer_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-03-10T07:27:39,681 copying build/lib/colossalai/kernel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-03-10T07:27:39,686 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,689 copying build/lib/colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,694 copying build/lib/colossalai/kernel/op_builder/fused_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,698 copying build/lib/colossalai/kernel/op_builder/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,703 copying build/lib/colossalai/kernel/op_builder/multi_head_attn.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,707 copying build/lib/colossalai/kernel/op_builder/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,711 copying build/lib/colossalai/kernel/op_builder/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,715 copying build/lib/colossalai/kernel/op_builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,719 copying build/lib/colossalai/kernel/op_builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,724 copying build/lib/colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,728 copying build/lib/colossalai/kernel/op_builder/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-03-10T07:27:39,733 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T07:27:39,735 copying build/lib/colossalai/pipeline/pipeline_process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T07:27:39,740 copying build/lib/colossalai/pipeline/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T07:27:39,746 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T07:27:39,748 copying build/lib/colossalai/pipeline/rpc/_pipeline_base.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T07:27:39,755 copying build/lib/colossalai/pipeline/rpc/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T07:27:39,760 copying build/lib/colossalai/pipeline/rpc/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T07:27:39,764 copying build/lib/colossalai/pipeline/rpc/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-03-10T07:27:39,768 copying build/lib/colossalai/pipeline/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T07:27:39,771 copying build/lib/colossalai/pipeline/pipelinable.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T07:27:39,777 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-03-10T07:27:39,779 copying build/lib/colossalai/pipeline/middleware/topo.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-03-10T07:27:39,785 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-03-10T07:27:39,787 copying build/lib/colossalai/pipeline/middleware/adaptor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-03-10T07:27:39,792 copying build/lib/colossalai/pipeline/middleware/adaptor/fx.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-03-10T07:27:39,796 copying build/lib/colossalai/pipeline/middleware/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-03-10T07:27:39,800 copying build/lib/colossalai/pipeline/layer_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-03-10T07:27:39,805 creating build/bdist.linux-armv7l/wheel/colossalai/engine 2023-03-10T07:27:39,808 creating build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-03-10T07:27:39,810 copying build/lib/colossalai/engine/gradient_accumulation/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-03-10T07:27:39,814 copying build/lib/colossalai/engine/gradient_accumulation/_gradient_accumulation.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-03-10T07:27:39,818 copying build/lib/colossalai/engine/_base_engine.py -> build/bdist.linux-armv7l/wheel/colossalai/engine 2023-03-10T07:27:39,824 creating build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T07:27:39,826 copying build/lib/colossalai/engine/schedule/_pipeline_schedule_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T07:27:39,830 copying build/lib/colossalai/engine/schedule/_base_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T07:27:39,835 copying build/lib/colossalai/engine/schedule/_non_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T07:27:39,838 copying build/lib/colossalai/engine/schedule/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T07:27:39,845 copying build/lib/colossalai/engine/schedule/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-03-10T07:27:39,849 creating build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,851 copying build/lib/colossalai/engine/gradient_handler/_moe_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,855 copying build/lib/colossalai/engine/gradient_handler/_zero_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,858 copying build/lib/colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,862 copying build/lib/colossalai/engine/gradient_handler/_base_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,865 copying build/lib/colossalai/engine/gradient_handler/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,869 copying build/lib/colossalai/engine/gradient_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,873 copying build/lib/colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,876 copying build/lib/colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-03-10T07:27:39,880 copying build/lib/colossalai/engine/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine 2023-03-10T07:27:39,884 creating build/bdist.linux-armv7l/wheel/colossalai/_C 2023-03-10T07:27:39,886 copying build/lib/colossalai/_C/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_C 2023-03-10T07:27:39,889 copying build/lib/colossalai/core.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T07:27:39,892 copying build/lib/colossalai/constants.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T07:27:39,897 creating build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T07:27:39,899 copying build/lib/colossalai/context/moe_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T07:27:39,904 creating build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T07:27:39,906 copying build/lib/colossalai/context/random/seed_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T07:27:39,910 copying build/lib/colossalai/context/random/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T07:27:39,913 copying build/lib/colossalai/context/random/_helper.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-03-10T07:27:39,917 copying build/lib/colossalai/context/singleton_meta.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T07:27:39,921 copying build/lib/colossalai/context/parallel_mode.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T07:27:39,926 creating build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,927 copying build/lib/colossalai/context/process_group_initializer/initializer_data.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,931 copying build/lib/colossalai/context/process_group_initializer/initializer_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,936 copying build/lib/colossalai/context/process_group_initializer/initializer_pipeline.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,939 copying build/lib/colossalai/context/process_group_initializer/initializer_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,944 copying build/lib/colossalai/context/process_group_initializer/initializer_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,949 copying build/lib/colossalai/context/process_group_initializer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,953 copying build/lib/colossalai/context/process_group_initializer/initializer_model.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,957 copying build/lib/colossalai/context/process_group_initializer/initializer_sequence.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,962 copying build/lib/colossalai/context/process_group_initializer/process_group_initializer.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,966 copying build/lib/colossalai/context/process_group_initializer/initializer_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,970 copying build/lib/colossalai/context/process_group_initializer/initializer_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-03-10T07:27:39,974 copying build/lib/colossalai/context/config.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T07:27:39,978 copying build/lib/colossalai/context/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T07:27:39,982 copying build/lib/colossalai/context/parallel_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-03-10T07:27:39,990 creating build/bdist.linux-armv7l/wheel/colossalai/zero 2023-03-10T07:27:39,993 creating build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-03-10T07:27:39,995 copying build/lib/colossalai/zero/init_ctx/init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-03-10T07:27:40,001 copying build/lib/colossalai/zero/init_ctx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-03-10T07:27:40,006 creating build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T07:27:40,008 copying build/lib/colossalai/zero/shard_utils/base_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T07:27:40,012 copying build/lib/colossalai/zero/shard_utils/commons.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T07:27:40,016 copying build/lib/colossalai/zero/shard_utils/tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T07:27:40,020 copying build/lib/colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T07:27:40,024 copying build/lib/colossalai/zero/shard_utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-03-10T07:27:40,030 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T07:27:40,032 copying build/lib/colossalai/zero/sharded_model/sharded_model_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T07:27:40,038 copying build/lib/colossalai/zero/sharded_model/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T07:27:40,042 copying build/lib/colossalai/zero/sharded_model/reduce_scatter.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T07:27:40,047 copying build/lib/colossalai/zero/sharded_model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T07:27:40,051 copying build/lib/colossalai/zero/sharded_model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-03-10T07:27:40,056 creating build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T07:27:40,057 copying build/lib/colossalai/zero/utils/gemini_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T07:27:40,061 copying build/lib/colossalai/zero/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T07:27:40,065 copying build/lib/colossalai/zero/utils/zero_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-03-10T07:27:40,069 copying build/lib/colossalai/zero/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero 2023-03-10T07:27:40,073 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T07:27:40,075 copying build/lib/colossalai/zero/sharded_param/sharded_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T07:27:40,079 copying build/lib/colossalai/zero/sharded_param/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T07:27:40,082 copying build/lib/colossalai/zero/sharded_param/sharded_param.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-03-10T07:27:40,087 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T07:27:40,089 copying build/lib/colossalai/zero/sharded_optim/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T07:27:40,094 copying build/lib/colossalai/zero/sharded_optim/sharded_optim_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T07:27:40,100 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:40,103 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/base_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:40,107 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/bucket_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:40,111 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/parameter_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:40,116 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:40,120 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/gradient_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:40,124 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-03-10T07:27:40,128 copying build/lib/colossalai/zero/sharded_optim/low_level_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T07:27:40,134 copying build/lib/colossalai/zero/sharded_optim/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-03-10T07:27:40,138 copying build/lib/colossalai/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-03-10T07:27:40,145 creating build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,148 copying build/lib/op_builder/scaled_upper_triangle_masked_softmax.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,152 copying build/lib/op_builder/fused_optim.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,156 copying build/lib/op_builder/cpu_adam.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,160 copying build/lib/op_builder/multi_head_attn.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,165 copying build/lib/op_builder/moe.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,169 copying build/lib/op_builder/utils.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,174 copying build/lib/op_builder/__init__.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,178 copying build/lib/op_builder/builder.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,183 copying build/lib/op_builder/scaled_masked_softmax.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,187 copying build/lib/op_builder/layernorm.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-03-10T07:27:40,191 running install_egg_info 2023-03-10T07:27:40,336 running egg_info 2023-03-10T07:27:40,391 writing colossalai.egg-info/PKG-INFO 2023-03-10T07:27:40,397 writing dependency_links to colossalai.egg-info/dependency_links.txt 2023-03-10T07:27:40,401 writing entry points to colossalai.egg-info/entry_points.txt 2023-03-10T07:27:40,404 writing requirements to colossalai.egg-info/requires.txt 2023-03-10T07:27:40,407 writing top-level names to colossalai.egg-info/top_level.txt 2023-03-10T07:27:40,691 reading manifest file 'colossalai.egg-info/SOURCES.txt' 2023-03-10T07:27:40,761 reading manifest template 'MANIFEST.in' 2023-03-10T07:27:42,747 warning: no files found matching '*.tr' under directory 'colossalai' 2023-03-10T07:27:43,750 warning: no files found matching '*.cc' under directory 'colossalai' 2023-03-10T07:27:44,258 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-03-10T07:27:44,276 adding license file 'LICENSE' 2023-03-10T07:27:44,371 writing manifest file 'colossalai.egg-info/SOURCES.txt' 2023-03-10T07:27:44,376 Copying colossalai.egg-info to build/bdist.linux-armv7l/wheel/colossalai-0.2.7-py3.7.egg-info 2023-03-10T07:27:44,403 running install_scripts 2023-03-10T07:27:44,451 creating build/bdist.linux-armv7l/wheel/colossalai-0.2.7.dist-info/WHEEL 2023-03-10T07:27:44,456 creating '/tmp/pip-wheel-c5zr2w2o/colossalai-0.2.7-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2023-03-10T07:27:44,462 adding 'colossalai/__init__.py' 2023-03-10T07:27:44,466 adding 'colossalai/constants.py' 2023-03-10T07:27:44,470 adding 'colossalai/core.py' 2023-03-10T07:27:44,475 adding 'colossalai/global_variables.py' 2023-03-10T07:27:44,483 adding 'colossalai/initialize.py' 2023-03-10T07:27:44,488 adding 'colossalai/version.py' 2023-03-10T07:27:44,495 adding 'colossalai/_C/__init__.py' 2023-03-10T07:27:44,499 adding 'colossalai/amp/__init__.py' 2023-03-10T07:27:44,502 adding 'colossalai/amp/amp_type.py' 2023-03-10T07:27:44,505 adding 'colossalai/amp/apex_amp/__init__.py' 2023-03-10T07:27:44,508 adding 'colossalai/amp/apex_amp/apex_amp.py' 2023-03-10T07:27:44,512 adding 'colossalai/amp/naive_amp/__init__.py' 2023-03-10T07:27:44,517 adding 'colossalai/amp/naive_amp/_fp16_optimizer.py' 2023-03-10T07:27:44,520 adding 'colossalai/amp/naive_amp/_utils.py' 2023-03-10T07:27:44,523 adding 'colossalai/amp/naive_amp/naive_amp.py' 2023-03-10T07:27:44,528 adding 'colossalai/amp/naive_amp/grad_scaler/__init__.py' 2023-03-10T07:27:44,531 adding 'colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py' 2023-03-10T07:27:44,534 adding 'colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py' 2023-03-10T07:27:44,537 adding 'colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py' 2023-03-10T07:27:44,542 adding 'colossalai/amp/torch_amp/__init__.py' 2023-03-10T07:27:44,551 adding 'colossalai/amp/torch_amp/_grad_scaler.py' 2023-03-10T07:27:44,556 adding 'colossalai/amp/torch_amp/torch_amp.py' 2023-03-10T07:27:44,561 adding 'colossalai/auto_parallel/__init__.py' 2023-03-10T07:27:44,568 adding 'colossalai/auto_parallel/checkpoint/__init__.py' 2023-03-10T07:27:44,572 adding 'colossalai/auto_parallel/checkpoint/build_c_ext.py' 2023-03-10T07:27:44,579 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_base.py' 2023-03-10T07:27:44,585 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py' 2023-03-10T07:27:44,594 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py' 2023-03-10T07:27:44,600 adding 'colossalai/auto_parallel/checkpoint/operation.py' 2023-03-10T07:27:44,608 adding 'colossalai/auto_parallel/meta_profiler/__init__.py' 2023-03-10T07:27:44,612 adding 'colossalai/auto_parallel/meta_profiler/constants.py' 2023-03-10T07:27:44,616 adding 'colossalai/auto_parallel/meta_profiler/metainfo.py' 2023-03-10T07:27:44,620 adding 'colossalai/auto_parallel/meta_profiler/registry.py' 2023-03-10T07:27:44,627 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py' 2023-03-10T07:27:44,632 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/activation.py' 2023-03-10T07:27:44,638 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py' 2023-03-10T07:27:44,645 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/conv.py' 2023-03-10T07:27:44,649 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/embedding.py' 2023-03-10T07:27:44,657 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/linear.py' 2023-03-10T07:27:44,664 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/non_spmd.py' 2023-03-10T07:27:44,670 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/norm.py' 2023-03-10T07:27:44,676 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py' 2023-03-10T07:27:44,682 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/tensor.py' 2023-03-10T07:27:44,688 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/where.py' 2023-03-10T07:27:44,696 adding 'colossalai/auto_parallel/passes/__init__.py' 2023-03-10T07:27:44,702 adding 'colossalai/auto_parallel/passes/comm_metainfo_pass.py' 2023-03-10T07:27:44,708 adding 'colossalai/auto_parallel/passes/constants.py' 2023-03-10T07:27:44,714 adding 'colossalai/auto_parallel/passes/meta_info_prop.py' 2023-03-10T07:27:44,722 adding 'colossalai/auto_parallel/passes/runtime_apply_pass.py' 2023-03-10T07:27:44,731 adding 'colossalai/auto_parallel/passes/runtime_preparation_pass.py' 2023-03-10T07:27:44,737 adding 'colossalai/auto_parallel/pipeline_shard/__init__.py' 2023-03-10T07:27:44,741 adding 'colossalai/auto_parallel/tensor_shard/__init__.py' 2023-03-10T07:27:44,746 adding 'colossalai/auto_parallel/tensor_shard/constants.py' 2023-03-10T07:27:44,754 adding 'colossalai/auto_parallel/tensor_shard/initialize.py' 2023-03-10T07:27:44,760 adding 'colossalai/auto_parallel/tensor_shard/options.py' 2023-03-10T07:27:44,767 adding 'colossalai/auto_parallel/tensor_shard/sharding_strategy.py' 2023-03-10T07:27:44,780 adding 'colossalai/auto_parallel/tensor_shard/node_handler/__init__.py' 2023-03-10T07:27:44,786 adding 'colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py' 2023-03-10T07:27:44,793 adding 'colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py' 2023-03-10T07:27:44,796 adding 'colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py' 2023-03-10T07:27:44,799 adding 'colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py' 2023-03-10T07:27:44,805 adding 'colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py' 2023-03-10T07:27:44,811 adding 'colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py' 2023-03-10T07:27:44,819 adding 'colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py' 2023-03-10T07:27:44,825 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py' 2023-03-10T07:27:44,828 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py' 2023-03-10T07:27:44,832 adding 'colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py' 2023-03-10T07:27:44,839 adding 'colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py' 2023-03-10T07:27:44,846 adding 'colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py' 2023-03-10T07:27:44,851 adding 'colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py' 2023-03-10T07:27:44,854 adding 'colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py' 2023-03-10T07:27:44,856 adding 'colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py' 2023-03-10T07:27:44,859 adding 'colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py' 2023-03-10T07:27:44,863 adding 'colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py' 2023-03-10T07:27:44,866 adding 'colossalai/auto_parallel/tensor_shard/node_handler/registry.py' 2023-03-10T07:27:44,869 adding 'colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py' 2023-03-10T07:27:44,871 adding 'colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py' 2023-03-10T07:27:44,875 adding 'colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py' 2023-03-10T07:27:44,880 adding 'colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py' 2023-03-10T07:27:44,885 adding 'colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py' 2023-03-10T07:27:44,890 adding 'colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py' 2023-03-10T07:27:44,892 adding 'colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py' 2023-03-10T07:27:44,895 adding 'colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py' 2023-03-10T07:27:44,901 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py' 2023-03-10T07:27:44,906 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py' 2023-03-10T07:27:44,910 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py' 2023-03-10T07:27:44,914 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py' 2023-03-10T07:27:44,919 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py' 2023-03-10T07:27:44,923 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py' 2023-03-10T07:27:44,928 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py' 2023-03-10T07:27:44,933 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py' 2023-03-10T07:27:44,941 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py' 2023-03-10T07:27:44,947 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py' 2023-03-10T07:27:44,951 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py' 2023-03-10T07:27:44,955 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py' 2023-03-10T07:27:44,961 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py' 2023-03-10T07:27:44,965 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py' 2023-03-10T07:27:44,971 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py' 2023-03-10T07:27:44,976 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py' 2023-03-10T07:27:44,981 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py' 2023-03-10T07:27:44,984 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py' 2023-03-10T07:27:44,988 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py' 2023-03-10T07:27:44,996 adding 'colossalai/auto_parallel/tensor_shard/solver/__init__.py' 2023-03-10T07:27:45,003 adding 'colossalai/auto_parallel/tensor_shard/solver/cost_graph.py' 2023-03-10T07:27:45,008 adding 'colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py' 2023-03-10T07:27:45,016 adding 'colossalai/auto_parallel/tensor_shard/solver/solver.py' 2023-03-10T07:27:45,023 adding 'colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py' 2023-03-10T07:27:45,028 adding 'colossalai/auto_parallel/tensor_shard/utils/__init__.py' 2023-03-10T07:27:45,032 adding 'colossalai/auto_parallel/tensor_shard/utils/broadcast.py' 2023-03-10T07:27:45,039 adding 'colossalai/auto_parallel/tensor_shard/utils/factory.py' 2023-03-10T07:27:45,044 adding 'colossalai/auto_parallel/tensor_shard/utils/misc.py' 2023-03-10T07:27:45,049 adding 'colossalai/auto_parallel/tensor_shard/utils/reshape.py' 2023-03-10T07:27:45,054 adding 'colossalai/auto_parallel/tensor_shard/utils/sharding.py' 2023-03-10T07:27:45,064 adding 'colossalai/booster/__init__.py' 2023-03-10T07:27:45,067 adding 'colossalai/booster/accelerator.py' 2023-03-10T07:27:45,070 adding 'colossalai/booster/booster.py' 2023-03-10T07:27:45,074 adding 'colossalai/booster/environment_table.py' 2023-03-10T07:27:45,079 adding 'colossalai/booster/plugin.py' 2023-03-10T07:27:45,086 adding 'colossalai/booster/precision.py' 2023-03-10T07:27:45,090 adding 'colossalai/builder/__init__.py' 2023-03-10T07:27:45,094 adding 'colossalai/builder/builder.py' 2023-03-10T07:27:45,101 adding 'colossalai/cli/__init__.py' 2023-03-10T07:27:45,108 adding 'colossalai/cli/cli.py' 2023-03-10T07:27:45,118 adding 'colossalai/cli/benchmark/__init__.py' 2023-03-10T07:27:45,124 adding 'colossalai/cli/benchmark/benchmark.py' 2023-03-10T07:27:45,129 adding 'colossalai/cli/benchmark/models.py' 2023-03-10T07:27:45,134 adding 'colossalai/cli/benchmark/utils.py' 2023-03-10T07:27:45,140 adding 'colossalai/cli/check/__init__.py' 2023-03-10T07:27:45,144 adding 'colossalai/cli/check/check_installation.py' 2023-03-10T07:27:45,151 adding 'colossalai/cli/launcher/__init__.py' 2023-03-10T07:27:45,156 adding 'colossalai/cli/launcher/hostinfo.py' 2023-03-10T07:27:45,161 adding 'colossalai/cli/launcher/multinode_runner.py' 2023-03-10T07:27:45,167 adding 'colossalai/cli/launcher/run.py' 2023-03-10T07:27:45,172 adding 'colossalai/communication/__init__.py' 2023-03-10T07:27:45,177 adding 'colossalai/communication/collective.py' 2023-03-10T07:27:45,182 adding 'colossalai/communication/p2p.py' 2023-03-10T07:27:45,187 adding 'colossalai/communication/p2p_v2.py' 2023-03-10T07:27:45,190 adding 'colossalai/communication/ring.py' 2023-03-10T07:27:45,194 adding 'colossalai/communication/utils.py' 2023-03-10T07:27:45,199 adding 'colossalai/context/__init__.py' 2023-03-10T07:27:45,202 adding 'colossalai/context/config.py' 2023-03-10T07:27:45,205 adding 'colossalai/context/moe_context.py' 2023-03-10T07:27:45,212 adding 'colossalai/context/parallel_context.py' 2023-03-10T07:27:45,215 adding 'colossalai/context/parallel_mode.py' 2023-03-10T07:27:45,219 adding 'colossalai/context/singleton_meta.py' 2023-03-10T07:27:45,229 adding 'colossalai/context/process_group_initializer/__init__.py' 2023-03-10T07:27:45,232 adding 'colossalai/context/process_group_initializer/initializer_1d.py' 2023-03-10T07:27:45,235 adding 'colossalai/context/process_group_initializer/initializer_2d.py' 2023-03-10T07:27:45,240 adding 'colossalai/context/process_group_initializer/initializer_2p5d.py' 2023-03-10T07:27:45,245 adding 'colossalai/context/process_group_initializer/initializer_3d.py' 2023-03-10T07:27:45,250 adding 'colossalai/context/process_group_initializer/initializer_data.py' 2023-03-10T07:27:45,256 adding 'colossalai/context/process_group_initializer/initializer_model.py' 2023-03-10T07:27:45,262 adding 'colossalai/context/process_group_initializer/initializer_pipeline.py' 2023-03-10T07:27:45,267 adding 'colossalai/context/process_group_initializer/initializer_sequence.py' 2023-03-10T07:27:45,273 adding 'colossalai/context/process_group_initializer/initializer_tensor.py' 2023-03-10T07:27:45,276 adding 'colossalai/context/process_group_initializer/process_group_initializer.py' 2023-03-10T07:27:45,280 adding 'colossalai/context/random/__init__.py' 2023-03-10T07:27:45,283 adding 'colossalai/context/random/_helper.py' 2023-03-10T07:27:45,286 adding 'colossalai/context/random/seed_manager.py' 2023-03-10T07:27:45,289 adding 'colossalai/device/__init__.py' 2023-03-10T07:27:45,295 adding 'colossalai/device/alpha_beta_profiler.py' 2023-03-10T07:27:45,298 adding 'colossalai/device/calc_pipeline_strategy.py' 2023-03-10T07:27:45,302 adding 'colossalai/device/device_mesh.py' 2023-03-10T07:27:45,306 adding 'colossalai/engine/__init__.py' 2023-03-10T07:27:45,310 adding 'colossalai/engine/_base_engine.py' 2023-03-10T07:27:45,313 adding 'colossalai/engine/gradient_accumulation/__init__.py' 2023-03-10T07:27:45,317 adding 'colossalai/engine/gradient_accumulation/_gradient_accumulation.py' 2023-03-10T07:27:45,321 adding 'colossalai/engine/gradient_handler/__init__.py' 2023-03-10T07:27:45,324 adding 'colossalai/engine/gradient_handler/_base_gradient_handler.py' 2023-03-10T07:27:45,326 adding 'colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py' 2023-03-10T07:27:45,329 adding 'colossalai/engine/gradient_handler/_moe_gradient_handler.py' 2023-03-10T07:27:45,332 adding 'colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py' 2023-03-10T07:27:45,334 adding 'colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py' 2023-03-10T07:27:45,337 adding 'colossalai/engine/gradient_handler/_zero_gradient_handler.py' 2023-03-10T07:27:45,339 adding 'colossalai/engine/gradient_handler/utils.py' 2023-03-10T07:27:45,343 adding 'colossalai/engine/schedule/__init__.py' 2023-03-10T07:27:45,346 adding 'colossalai/engine/schedule/_base_schedule.py' 2023-03-10T07:27:45,349 adding 'colossalai/engine/schedule/_non_pipeline_schedule.py' 2023-03-10T07:27:45,359 adding 'colossalai/engine/schedule/_pipeline_schedule.py' 2023-03-10T07:27:45,363 adding 'colossalai/engine/schedule/_pipeline_schedule_v2.py' 2023-03-10T07:27:45,367 adding 'colossalai/fx/__init__.py' 2023-03-10T07:27:45,371 adding 'colossalai/fx/_compatibility.py' 2023-03-10T07:27:45,376 adding 'colossalai/fx/_meta_registrations.py' 2023-03-10T07:27:45,380 adding 'colossalai/fx/graph_module.py' 2023-03-10T07:27:45,384 adding 'colossalai/fx/proxy.py' 2023-03-10T07:27:45,388 adding 'colossalai/fx/codegen/__init__.py' 2023-03-10T07:27:45,400 adding 'colossalai/fx/codegen/activation_checkpoint_codegen.py' 2023-03-10T07:27:45,405 adding 'colossalai/fx/passes/__init__.py' 2023-03-10T07:27:45,410 adding 'colossalai/fx/passes/adding_split_node_pass.py' 2023-03-10T07:27:45,415 adding 'colossalai/fx/passes/concrete_info_prop.py' 2023-03-10T07:27:45,421 adding 'colossalai/fx/passes/meta_info_prop.py' 2023-03-10T07:27:45,427 adding 'colossalai/fx/passes/passes_for_gpt2_test.py' 2023-03-10T07:27:45,432 adding 'colossalai/fx/passes/shard_1d_pass.py' 2023-03-10T07:27:45,437 adding 'colossalai/fx/passes/split_module.py' 2023-03-10T07:27:45,441 adding 'colossalai/fx/passes/utils.py' 2023-03-10T07:27:45,446 adding 'colossalai/fx/profiler/__init__.py' 2023-03-10T07:27:45,449 adding 'colossalai/fx/profiler/constants.py' 2023-03-10T07:27:45,453 adding 'colossalai/fx/profiler/dataflow.py' 2023-03-10T07:27:45,457 adding 'colossalai/fx/profiler/memory_utils.py' 2023-03-10T07:27:45,462 adding 'colossalai/fx/profiler/opcount.py' 2023-03-10T07:27:45,467 adding 'colossalai/fx/profiler/profiler.py' 2023-03-10T07:27:45,471 adding 'colossalai/fx/profiler/shard_utils.py' 2023-03-10T07:27:45,475 adding 'colossalai/fx/profiler/tensor.py' 2023-03-10T07:27:45,479 adding 'colossalai/fx/profiler/experimental/__init__.py' 2023-03-10T07:27:45,482 adding 'colossalai/fx/profiler/experimental/constants.py' 2023-03-10T07:27:45,486 adding 'colossalai/fx/profiler/experimental/profiler.py' 2023-03-10T07:27:45,490 adding 'colossalai/fx/profiler/experimental/registry.py' 2023-03-10T07:27:45,493 adding 'colossalai/fx/profiler/experimental/shard_utils.py' 2023-03-10T07:27:45,497 adding 'colossalai/fx/profiler/experimental/profiler_function/__init__.py' 2023-03-10T07:27:45,501 adding 'colossalai/fx/profiler/experimental/profiler_function/activation_function.py' 2023-03-10T07:27:45,504 adding 'colossalai/fx/profiler/experimental/profiler_function/arithmetic.py' 2023-03-10T07:27:45,507 adding 'colossalai/fx/profiler/experimental/profiler_function/embedding.py' 2023-03-10T07:27:45,510 adding 'colossalai/fx/profiler/experimental/profiler_function/linear.py' 2023-03-10T07:27:45,514 adding 'colossalai/fx/profiler/experimental/profiler_function/normalization.py' 2023-03-10T07:27:45,517 adding 'colossalai/fx/profiler/experimental/profiler_function/pooling.py' 2023-03-10T07:27:45,520 adding 'colossalai/fx/profiler/experimental/profiler_function/python_ops.py' 2023-03-10T07:27:45,523 adding 'colossalai/fx/profiler/experimental/profiler_function/torch_ops.py' 2023-03-10T07:27:45,528 adding 'colossalai/fx/profiler/experimental/profiler_module/__init__.py' 2023-03-10T07:27:45,531 adding 'colossalai/fx/profiler/experimental/profiler_module/activation_function.py' 2023-03-10T07:27:45,534 adding 'colossalai/fx/profiler/experimental/profiler_module/attention.py' 2023-03-10T07:27:45,538 adding 'colossalai/fx/profiler/experimental/profiler_module/convolution.py' 2023-03-10T07:27:45,541 adding 'colossalai/fx/profiler/experimental/profiler_module/dropout.py' 2023-03-10T07:27:45,544 adding 'colossalai/fx/profiler/experimental/profiler_module/embedding.py' 2023-03-10T07:27:45,547 adding 'colossalai/fx/profiler/experimental/profiler_module/linear.py' 2023-03-10T07:27:45,551 adding 'colossalai/fx/profiler/experimental/profiler_module/normalization.py' 2023-03-10T07:27:45,554 adding 'colossalai/fx/profiler/experimental/profiler_module/pooling.py' 2023-03-10T07:27:45,557 adding 'colossalai/fx/profiler/experimental/profiler_module/rnn.py' 2023-03-10T07:27:45,560 adding 'colossalai/fx/profiler/experimental/profiler_module/torch_op.py' 2023-03-10T07:27:45,565 adding 'colossalai/fx/tracer/__init__.py' 2023-03-10T07:27:45,568 adding 'colossalai/fx/tracer/_meta_trace.py' 2023-03-10T07:27:45,572 adding 'colossalai/fx/tracer/_symbolic_trace.py' 2023-03-10T07:27:45,575 adding 'colossalai/fx/tracer/_tracer_utils.py' 2023-03-10T07:27:45,583 adding 'colossalai/fx/tracer/experimental.py' 2023-03-10T07:27:45,587 adding 'colossalai/fx/tracer/registry.py' 2023-03-10T07:27:45,595 adding 'colossalai/fx/tracer/tracer.py' 2023-03-10T07:27:45,599 adding 'colossalai/fx/tracer/bias_addition_patch/__init__.py' 2023-03-10T07:27:45,604 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py' 2023-03-10T07:27:45,607 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py' 2023-03-10T07:27:45,611 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py' 2023-03-10T07:27:45,614 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py' 2023-03-10T07:27:45,618 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py' 2023-03-10T07:27:45,622 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py' 2023-03-10T07:27:45,625 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py' 2023-03-10T07:27:45,629 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py' 2023-03-10T07:27:45,632 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py' 2023-03-10T07:27:45,636 adding 'colossalai/fx/tracer/meta_patch/__init__.py' 2023-03-10T07:27:45,640 adding 'colossalai/fx/tracer/meta_patch/patched_function/__init__.py' 2023-03-10T07:27:45,643 adding 'colossalai/fx/tracer/meta_patch/patched_function/activation_function.py' 2023-03-10T07:27:45,646 adding 'colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py' 2023-03-10T07:27:45,650 adding 'colossalai/fx/tracer/meta_patch/patched_function/convolution.py' 2023-03-10T07:27:45,653 adding 'colossalai/fx/tracer/meta_patch/patched_function/embedding.py' 2023-03-10T07:27:45,656 adding 'colossalai/fx/tracer/meta_patch/patched_function/normalization.py' 2023-03-10T07:27:45,660 adding 'colossalai/fx/tracer/meta_patch/patched_function/python_ops.py' 2023-03-10T07:27:45,664 adding 'colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py' 2023-03-10T07:27:45,669 adding 'colossalai/fx/tracer/meta_patch/patched_module/__init__.py' 2023-03-10T07:27:45,672 adding 'colossalai/fx/tracer/meta_patch/patched_module/activation_function.py' 2023-03-10T07:27:45,675 adding 'colossalai/fx/tracer/meta_patch/patched_module/convolution.py' 2023-03-10T07:27:45,678 adding 'colossalai/fx/tracer/meta_patch/patched_module/embedding.py' 2023-03-10T07:27:45,682 adding 'colossalai/fx/tracer/meta_patch/patched_module/linear.py' 2023-03-10T07:27:45,685 adding 'colossalai/fx/tracer/meta_patch/patched_module/normalization.py' 2023-03-10T07:27:45,688 adding 'colossalai/fx/tracer/meta_patch/patched_module/pooling.py' 2023-03-10T07:27:45,692 adding 'colossalai/fx/tracer/meta_patch/patched_module/rnn.py' 2023-03-10T07:27:45,699 adding 'colossalai/gemini/__init__.py' 2023-03-10T07:27:45,703 adding 'colossalai/gemini/gemini_context.py' 2023-03-10T07:27:45,707 adding 'colossalai/gemini/gemini_mgr.py' 2023-03-10T07:27:45,711 adding 'colossalai/gemini/placement_policy.py' 2023-03-10T07:27:45,715 adding 'colossalai/gemini/stateful_tensor.py' 2023-03-10T07:27:45,720 adding 'colossalai/gemini/stateful_tensor_mgr.py' 2023-03-10T07:27:45,724 adding 'colossalai/gemini/tensor_placement_policy.py' 2023-03-10T07:27:45,727 adding 'colossalai/gemini/tensor_utils.py' 2023-03-10T07:27:45,732 adding 'colossalai/gemini/chunk/__init__.py' 2023-03-10T07:27:45,742 adding 'colossalai/gemini/chunk/chunk.py' 2023-03-10T07:27:45,747 adding 'colossalai/gemini/chunk/manager.py' 2023-03-10T07:27:45,752 adding 'colossalai/gemini/chunk/search_utils.py' 2023-03-10T07:27:45,755 adding 'colossalai/gemini/chunk/utils.py' 2023-03-10T07:27:45,760 adding 'colossalai/gemini/memory_tracer/__init__.py' 2023-03-10T07:27:45,763 adding 'colossalai/gemini/memory_tracer/chunk_memstats_collector.py' 2023-03-10T07:27:45,767 adding 'colossalai/gemini/memory_tracer/memory_monitor.py' 2023-03-10T07:27:45,771 adding 'colossalai/gemini/memory_tracer/memory_stats.py' 2023-03-10T07:27:45,774 adding 'colossalai/gemini/memory_tracer/memstats_collector.py' 2023-03-10T07:27:45,777 adding 'colossalai/gemini/memory_tracer/param_runtime_order.py' 2023-03-10T07:27:45,781 adding 'colossalai/gemini/memory_tracer/runtime_mem_tracer.py' 2023-03-10T07:27:45,784 adding 'colossalai/gemini/memory_tracer/static_memstats_collector.py' 2023-03-10T07:27:45,787 adding 'colossalai/gemini/memory_tracer/utils.py' 2023-03-10T07:27:45,790 adding 'colossalai/gemini/ophooks/__init__.py' 2023-03-10T07:27:45,793 adding 'colossalai/gemini/ophooks/_shard_grad_ophook.py' 2023-03-10T07:27:45,795 adding 'colossalai/gemini/ophooks/_shard_param_ophook.py' 2023-03-10T07:27:45,798 adding 'colossalai/gemini/ophooks/runtime_mem_tracer_hook.py' 2023-03-10T07:27:45,801 adding 'colossalai/gemini/ophooks/utils.py' 2023-03-10T07:27:45,805 adding 'colossalai/gemini/paramhooks/__init__.py' 2023-03-10T07:27:45,808 adding 'colossalai/gemini/paramhooks/_param_hookmgr.py' 2023-03-10T07:27:45,811 adding 'colossalai/kernel/__init__.py' 2023-03-10T07:27:45,815 adding 'colossalai/kernel/cuda_native/__init__.py' 2023-03-10T07:27:45,820 adding 'colossalai/kernel/cuda_native/flash_attention.py' 2023-03-10T07:27:45,823 adding 'colossalai/kernel/cuda_native/layer_norm.py' 2023-03-10T07:27:45,827 adding 'colossalai/kernel/cuda_native/multihead_attention.py' 2023-03-10T07:27:45,831 adding 'colossalai/kernel/cuda_native/scaled_softmax.py' 2023-03-10T07:27:45,835 adding 'colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp' 2023-03-10T07:27:45,838 adding 'colossalai/kernel/cuda_native/csrc/compat.h' 2023-03-10T07:27:45,842 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.cpp' 2023-03-10T07:27:45,846 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.h' 2023-03-10T07:27:45,849 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp' 2023-03-10T07:27:45,856 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu' 2023-03-10T07:27:45,859 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda.cpp' 2023-03-10T07:27:45,865 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu' 2023-03-10T07:27:45,869 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu' 2023-03-10T07:27:45,872 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh' 2023-03-10T07:27:45,877 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu' 2023-03-10T07:27:45,881 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu' 2023-03-10T07:27:45,884 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu' 2023-03-10T07:27:45,888 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu' 2023-03-10T07:27:45,893 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp' 2023-03-10T07:27:45,896 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h' 2023-03-10T07:27:45,899 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp' 2023-03-10T07:27:45,904 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h' 2023-03-10T07:27:45,907 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu' 2023-03-10T07:27:45,910 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp' 2023-03-10T07:27:45,914 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h' 2023-03-10T07:27:45,918 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu' 2023-03-10T07:27:45,921 adding 'colossalai/kernel/cuda_native/csrc/type_shim.h' 2023-03-10T07:27:45,927 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h' 2023-03-10T07:27:45,930 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/context.h' 2023-03-10T07:27:45,933 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h' 2023-03-10T07:27:45,935 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h' 2023-03-10T07:27:45,938 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h' 2023-03-10T07:27:45,941 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h' 2023-03-10T07:27:45,943 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h' 2023-03-10T07:27:45,947 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h' 2023-03-10T07:27:45,949 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh' 2023-03-10T07:27:45,952 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h' 2023-03-10T07:27:45,955 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h' 2023-03-10T07:27:45,958 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h' 2023-03-10T07:27:45,961 adding 'colossalai/kernel/jit/__init__.py' 2023-03-10T07:27:45,964 adding 'colossalai/kernel/jit/bias_dropout_add.py' 2023-03-10T07:27:45,966 adding 'colossalai/kernel/jit/bias_gelu.py' 2023-03-10T07:27:45,969 adding 'colossalai/kernel/jit/option.py' 2023-03-10T07:27:45,973 adding 'colossalai/kernel/op_builder/__init__.py' 2023-03-10T07:27:45,977 adding 'colossalai/kernel/op_builder/builder.py' 2023-03-10T07:27:45,980 adding 'colossalai/kernel/op_builder/cpu_adam.py' 2023-03-10T07:27:45,983 adding 'colossalai/kernel/op_builder/fused_optim.py' 2023-03-10T07:27:45,985 adding 'colossalai/kernel/op_builder/layernorm.py' 2023-03-10T07:27:45,988 adding 'colossalai/kernel/op_builder/moe.py' 2023-03-10T07:27:45,991 adding 'colossalai/kernel/op_builder/multi_head_attn.py' 2023-03-10T07:27:45,993 adding 'colossalai/kernel/op_builder/scaled_masked_softmax.py' 2023-03-10T07:27:45,996 adding 'colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py' 2023-03-10T07:27:45,999 adding 'colossalai/kernel/op_builder/utils.py' 2023-03-10T07:27:46,003 adding 'colossalai/logging/__init__.py' 2023-03-10T07:27:46,007 adding 'colossalai/logging/logger.py' 2023-03-10T07:27:46,010 adding 'colossalai/nn/__init__.py' 2023-03-10T07:27:46,014 adding 'colossalai/nn/init.py' 2023-03-10T07:27:46,018 adding 'colossalai/nn/_ops/__init__.py' 2023-03-10T07:27:46,021 adding 'colossalai/nn/_ops/_utils.py' 2023-03-10T07:27:46,024 adding 'colossalai/nn/_ops/addmm.py' 2023-03-10T07:27:46,027 adding 'colossalai/nn/_ops/batch_norm.py' 2023-03-10T07:27:46,030 adding 'colossalai/nn/_ops/element_wise.py' 2023-03-10T07:27:46,034 adding 'colossalai/nn/_ops/embedding.py' 2023-03-10T07:27:46,037 adding 'colossalai/nn/_ops/embedding_bag.py' 2023-03-10T07:27:46,040 adding 'colossalai/nn/_ops/layernorm.py' 2023-03-10T07:27:46,043 adding 'colossalai/nn/_ops/linear.py' 2023-03-10T07:27:46,046 adding 'colossalai/nn/_ops/loss.py' 2023-03-10T07:27:46,049 adding 'colossalai/nn/_ops/view.py' 2023-03-10T07:27:46,053 adding 'colossalai/nn/layer/__init__.py' 2023-03-10T07:27:46,056 adding 'colossalai/nn/layer/base_layer.py' 2023-03-10T07:27:46,059 adding 'colossalai/nn/layer/colossalai_layer/__init__.py' 2023-03-10T07:27:46,062 adding 'colossalai/nn/layer/colossalai_layer/_utils.py' 2023-03-10T07:27:46,064 adding 'colossalai/nn/layer/colossalai_layer/dropout.py' 2023-03-10T07:27:46,068 adding 'colossalai/nn/layer/colossalai_layer/embedding.py' 2023-03-10T07:27:46,071 adding 'colossalai/nn/layer/colossalai_layer/linear.py' 2023-03-10T07:27:46,074 adding 'colossalai/nn/layer/colossalai_layer/normalization.py' 2023-03-10T07:27:46,077 adding 'colossalai/nn/layer/moe/__init__.py' 2023-03-10T07:27:46,081 adding 'colossalai/nn/layer/moe/_operation.py' 2023-03-10T07:27:46,084 adding 'colossalai/nn/layer/moe/experts.py' 2023-03-10T07:27:46,088 adding 'colossalai/nn/layer/moe/layers.py' 2023-03-10T07:27:46,092 adding 'colossalai/nn/layer/moe/routers.py' 2023-03-10T07:27:46,095 adding 'colossalai/nn/layer/moe/utils.py' 2023-03-10T07:27:46,098 adding 'colossalai/nn/layer/parallel_1d/__init__.py' 2023-03-10T07:27:46,101 adding 'colossalai/nn/layer/parallel_1d/_operation.py' 2023-03-10T07:27:46,104 adding 'colossalai/nn/layer/parallel_1d/_utils.py' 2023-03-10T07:27:46,113 adding 'colossalai/nn/layer/parallel_1d/layers.py' 2023-03-10T07:27:46,118 adding 'colossalai/nn/layer/parallel_2d/__init__.py' 2023-03-10T07:27:46,124 adding 'colossalai/nn/layer/parallel_2d/_operation.py' 2023-03-10T07:27:46,128 adding 'colossalai/nn/layer/parallel_2d/_utils.py' 2023-03-10T07:27:46,136 adding 'colossalai/nn/layer/parallel_2d/layers.py' 2023-03-10T07:27:46,141 adding 'colossalai/nn/layer/parallel_2p5d/__init__.py' 2023-03-10T07:27:46,148 adding 'colossalai/nn/layer/parallel_2p5d/_operation.py' 2023-03-10T07:27:46,151 adding 'colossalai/nn/layer/parallel_2p5d/_utils.py' 2023-03-10T07:27:46,160 adding 'colossalai/nn/layer/parallel_2p5d/layers.py' 2023-03-10T07:27:46,164 adding 'colossalai/nn/layer/parallel_3d/__init__.py' 2023-03-10T07:27:46,170 adding 'colossalai/nn/layer/parallel_3d/_operation.py' 2023-03-10T07:27:46,173 adding 'colossalai/nn/layer/parallel_3d/_utils.py' 2023-03-10T07:27:46,182 adding 'colossalai/nn/layer/parallel_3d/layers.py' 2023-03-10T07:27:46,186 adding 'colossalai/nn/layer/parallel_sequence/__init__.py' 2023-03-10T07:27:46,189 adding 'colossalai/nn/layer/parallel_sequence/_operation.py' 2023-03-10T07:27:46,192 adding 'colossalai/nn/layer/parallel_sequence/_utils.py' 2023-03-10T07:27:46,196 adding 'colossalai/nn/layer/parallel_sequence/layers.py' 2023-03-10T07:27:46,199 adding 'colossalai/nn/layer/utils/__init__.py' 2023-03-10T07:27:46,202 adding 'colossalai/nn/layer/utils/common.py' 2023-03-10T07:27:46,205 adding 'colossalai/nn/layer/vanilla/__init__.py' 2023-03-10T07:27:46,210 adding 'colossalai/nn/layer/vanilla/layers.py' 2023-03-10T07:27:46,214 adding 'colossalai/nn/layer/wrapper/__init__.py' 2023-03-10T07:27:46,216 adding 'colossalai/nn/layer/wrapper/pipeline_wrapper.py' 2023-03-10T07:27:46,220 adding 'colossalai/nn/loss/__init__.py' 2023-03-10T07:27:46,223 adding 'colossalai/nn/loss/loss_1d.py' 2023-03-10T07:27:46,227 adding 'colossalai/nn/loss/loss_2d.py' 2023-03-10T07:27:46,230 adding 'colossalai/nn/loss/loss_2p5d.py' 2023-03-10T07:27:46,233 adding 'colossalai/nn/loss/loss_3d.py' 2023-03-10T07:27:46,236 adding 'colossalai/nn/loss/loss_moe.py' 2023-03-10T07:27:46,240 adding 'colossalai/nn/lr_scheduler/__init__.py' 2023-03-10T07:27:46,243 adding 'colossalai/nn/lr_scheduler/cosine.py' 2023-03-10T07:27:46,247 adding 'colossalai/nn/lr_scheduler/delayed.py' 2023-03-10T07:27:46,249 adding 'colossalai/nn/lr_scheduler/linear.py' 2023-03-10T07:27:46,252 adding 'colossalai/nn/lr_scheduler/multistep.py' 2023-03-10T07:27:46,255 adding 'colossalai/nn/lr_scheduler/onecycle.py' 2023-03-10T07:27:46,258 adding 'colossalai/nn/lr_scheduler/poly.py' 2023-03-10T07:27:46,261 adding 'colossalai/nn/lr_scheduler/torch.py' 2023-03-10T07:27:46,265 adding 'colossalai/nn/metric/__init__.py' 2023-03-10T07:27:46,267 adding 'colossalai/nn/metric/_utils.py' 2023-03-10T07:27:46,270 adding 'colossalai/nn/metric/accuracy_2d.py' 2023-03-10T07:27:46,272 adding 'colossalai/nn/metric/accuracy_2p5d.py' 2023-03-10T07:27:46,275 adding 'colossalai/nn/metric/accuracy_3d.py' 2023-03-10T07:27:46,279 adding 'colossalai/nn/optimizer/__init__.py' 2023-03-10T07:27:46,281 adding 'colossalai/nn/optimizer/colossalai_optimizer.py' 2023-03-10T07:27:46,285 adding 'colossalai/nn/optimizer/cpu_adam.py' 2023-03-10T07:27:46,289 adding 'colossalai/nn/optimizer/fused_adam.py' 2023-03-10T07:27:46,292 adding 'colossalai/nn/optimizer/fused_lamb.py' 2023-03-10T07:27:46,296 adding 'colossalai/nn/optimizer/fused_sgd.py' 2023-03-10T07:27:46,299 adding 'colossalai/nn/optimizer/gemini_optimizer.py' 2023-03-10T07:27:46,302 adding 'colossalai/nn/optimizer/hybrid_adam.py' 2023-03-10T07:27:46,305 adding 'colossalai/nn/optimizer/lamb.py' 2023-03-10T07:27:46,308 adding 'colossalai/nn/optimizer/lars.py' 2023-03-10T07:27:46,312 adding 'colossalai/nn/optimizer/nvme_optimizer.py' 2023-03-10T07:27:46,317 adding 'colossalai/nn/optimizer/zero_optimizer.py' 2023-03-10T07:27:46,321 adding 'colossalai/nn/parallel/__init__.py' 2023-03-10T07:27:46,330 adding 'colossalai/nn/parallel/data_parallel.py' 2023-03-10T07:27:46,333 adding 'colossalai/nn/parallel/gemini_parallel.py' 2023-03-10T07:27:46,336 adding 'colossalai/nn/parallel/reducer.py' 2023-03-10T07:27:46,340 adding 'colossalai/nn/parallel/utils.py' 2023-03-10T07:27:46,343 adding 'colossalai/nn/parallel/zero_wrapper.py' 2023-03-10T07:27:46,347 adding 'colossalai/nn/parallel/layers/__init__.py' 2023-03-10T07:27:46,349 adding 'colossalai/nn/parallel/layers/colo_module.py' 2023-03-10T07:27:46,352 adding 'colossalai/nn/parallel/layers/embedding.py' 2023-03-10T07:27:46,354 adding 'colossalai/nn/parallel/layers/linear.py' 2023-03-10T07:27:46,358 adding 'colossalai/nn/parallel/layers/module_utils.py' 2023-03-10T07:27:46,361 adding 'colossalai/nn/parallel/layers/cache_embedding/__init__.py' 2023-03-10T07:27:46,364 adding 'colossalai/nn/parallel/layers/cache_embedding/base_embedding.py' 2023-03-10T07:27:46,372 adding 'colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py' 2023-03-10T07:27:46,376 adding 'colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py' 2023-03-10T07:27:46,379 adding 'colossalai/nn/parallel/layers/cache_embedding/copyer.py' 2023-03-10T07:27:46,382 adding 'colossalai/nn/parallel/layers/cache_embedding/embedding_config.py' 2023-03-10T07:27:46,386 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py' 2023-03-10T07:27:46,390 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py' 2023-03-10T07:27:46,393 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py' 2023-03-10T07:27:46,397 adding 'colossalai/pipeline/__init__.py' 2023-03-10T07:27:46,400 adding 'colossalai/pipeline/layer_spec.py' 2023-03-10T07:27:46,404 adding 'colossalai/pipeline/pipelinable.py' 2023-03-10T07:27:46,408 adding 'colossalai/pipeline/pipeline_process_group.py' 2023-03-10T07:27:46,412 adding 'colossalai/pipeline/utils.py' 2023-03-10T07:27:46,416 adding 'colossalai/pipeline/middleware/__init__.py' 2023-03-10T07:27:46,419 adding 'colossalai/pipeline/middleware/topo.py' 2023-03-10T07:27:46,423 adding 'colossalai/pipeline/middleware/adaptor/__init__.py' 2023-03-10T07:27:46,426 adding 'colossalai/pipeline/middleware/adaptor/fx.py' 2023-03-10T07:27:46,430 adding 'colossalai/pipeline/rpc/__init__.py' 2023-03-10T07:27:46,445 adding 'colossalai/pipeline/rpc/_pipeline_base.py' 2023-03-10T07:27:46,451 adding 'colossalai/pipeline/rpc/_pipeline_schedule.py' 2023-03-10T07:27:46,454 adding 'colossalai/pipeline/rpc/utils.py' 2023-03-10T07:27:46,459 adding 'colossalai/registry/__init__.py' 2023-03-10T07:27:46,462 adding 'colossalai/registry/registry.py' 2023-03-10T07:27:46,466 adding 'colossalai/tensor/__init__.py' 2023-03-10T07:27:46,469 adding 'colossalai/tensor/colo_parameter.py' 2023-03-10T07:27:46,474 adding 'colossalai/tensor/colo_tensor.py' 2023-03-10T07:27:46,480 adding 'colossalai/tensor/comm_spec.py' 2023-03-10T07:27:46,483 adding 'colossalai/tensor/compute_spec.py' 2023-03-10T07:27:46,485 adding 'colossalai/tensor/const.py' 2023-03-10T07:27:46,489 adding 'colossalai/tensor/dist_spec_mgr.py' 2023-03-10T07:27:46,492 adding 'colossalai/tensor/distspec.py' 2023-03-10T07:27:46,494 adding 'colossalai/tensor/op_wrapper.py' 2023-03-10T07:27:46,498 adding 'colossalai/tensor/param_op_hook.py' 2023-03-10T07:27:46,502 adding 'colossalai/tensor/process_group.py' 2023-03-10T07:27:46,510 adding 'colossalai/tensor/shape_consistency.py' 2023-03-10T07:27:46,515 adding 'colossalai/tensor/sharding_spec.py' 2023-03-10T07:27:46,518 adding 'colossalai/tensor/tensor_spec.py' 2023-03-10T07:27:46,522 adding 'colossalai/tensor/utils.py' 2023-03-10T07:27:46,526 adding 'colossalai/tensor/d_tensor/__init__.py' 2023-03-10T07:27:46,529 adding 'colossalai/tensor/d_tensor/comm_spec.py' 2023-03-10T07:27:46,533 adding 'colossalai/tensor/d_tensor/d_tensor.py' 2023-03-10T07:27:46,536 adding 'colossalai/tensor/d_tensor/layout.py' 2023-03-10T07:27:46,542 adding 'colossalai/tensor/d_tensor/layout_converter.py' 2023-03-10T07:27:46,545 adding 'colossalai/tensor/d_tensor/misc.py' 2023-03-10T07:27:46,549 adding 'colossalai/tensor/d_tensor/sharding_spec.py' 2023-03-10T07:27:46,552 adding 'colossalai/tensor/d_tensor/utils.py' 2023-03-10T07:27:46,555 adding 'colossalai/testing/__init__.py' 2023-03-10T07:27:46,558 adding 'colossalai/testing/comparison.py' 2023-03-10T07:27:46,561 adding 'colossalai/testing/pytest_wrapper.py' 2023-03-10T07:27:46,563 adding 'colossalai/testing/random.py' 2023-03-10T07:27:46,567 adding 'colossalai/testing/utils.py' 2023-03-10T07:27:46,570 adding 'colossalai/trainer/__init__.py' 2023-03-10T07:27:46,575 adding 'colossalai/trainer/_trainer.py' 2023-03-10T07:27:46,579 adding 'colossalai/trainer/hooks/__init__.py' 2023-03-10T07:27:46,582 adding 'colossalai/trainer/hooks/_base_hook.py' 2023-03-10T07:27:46,585 adding 'colossalai/trainer/hooks/_checkpoint_hook.py' 2023-03-10T07:27:46,587 adding 'colossalai/trainer/hooks/_commons_.py' 2023-03-10T07:27:46,591 adding 'colossalai/trainer/hooks/_log_hook.py' 2023-03-10T07:27:46,594 adding 'colossalai/trainer/hooks/_lr_scheduler_hook.py' 2023-03-10T07:27:46,599 adding 'colossalai/trainer/hooks/_metric_hook.py' 2023-03-10T07:27:46,603 adding 'colossalai/utils/__init__.py' 2023-03-10T07:27:46,607 adding 'colossalai/utils/activation_checkpoint.py' 2023-03-10T07:27:46,611 adding 'colossalai/utils/checkpointing.py' 2023-03-10T07:27:46,617 adding 'colossalai/utils/common.py' 2023-03-10T07:27:46,620 adding 'colossalai/utils/cuda.py' 2023-03-10T07:27:46,624 adding 'colossalai/utils/memory.py' 2023-03-10T07:27:46,627 adding 'colossalai/utils/moe.py' 2023-03-10T07:27:46,630 adding 'colossalai/utils/timer.py' 2023-03-10T07:27:46,633 adding 'colossalai/utils/checkpoint/__init__.py' 2023-03-10T07:27:46,636 adding 'colossalai/utils/checkpoint/module_checkpoint.py' 2023-03-10T07:27:46,639 adding 'colossalai/utils/checkpoint/utils.py' 2023-03-10T07:27:46,643 adding 'colossalai/utils/checkpoint_io/__init__.py' 2023-03-10T07:27:46,646 adding 'colossalai/utils/checkpoint_io/backend.py' 2023-03-10T07:27:46,648 adding 'colossalai/utils/checkpoint_io/constant.py' 2023-03-10T07:27:46,652 adding 'colossalai/utils/checkpoint_io/convertor.py' 2023-03-10T07:27:46,655 adding 'colossalai/utils/checkpoint_io/distributed.py' 2023-03-10T07:27:46,659 adding 'colossalai/utils/checkpoint_io/io.py' 2023-03-10T07:27:46,662 adding 'colossalai/utils/checkpoint_io/meta.py' 2023-03-10T07:27:46,665 adding 'colossalai/utils/checkpoint_io/reader.py' 2023-03-10T07:27:46,668 adding 'colossalai/utils/checkpoint_io/utils.py' 2023-03-10T07:27:46,672 adding 'colossalai/utils/checkpoint_io/writer.py' 2023-03-10T07:27:46,675 adding 'colossalai/utils/data_sampler/__init__.py' 2023-03-10T07:27:46,678 adding 'colossalai/utils/data_sampler/base_sampler.py' 2023-03-10T07:27:46,681 adding 'colossalai/utils/data_sampler/data_parallel_sampler.py' 2023-03-10T07:27:46,685 adding 'colossalai/utils/model/__init__.py' 2023-03-10T07:27:46,688 adding 'colossalai/utils/model/colo_init_context.py' 2023-03-10T07:27:46,694 adding 'colossalai/utils/model/experimental.py' 2023-03-10T07:27:46,698 adding 'colossalai/utils/model/lazy_init_context.py' 2023-03-10T07:27:46,701 adding 'colossalai/utils/model/utils.py' 2023-03-10T07:27:46,704 adding 'colossalai/utils/multi_tensor_apply/__init__.py' 2023-03-10T07:27:46,707 adding 'colossalai/utils/multi_tensor_apply/multi_tensor_apply.py' 2023-03-10T07:27:46,710 adding 'colossalai/utils/profiler/__init__.py' 2023-03-10T07:27:46,713 adding 'colossalai/utils/profiler/extention.py' 2023-03-10T07:27:46,717 adding 'colossalai/utils/profiler/profiler.py' 2023-03-10T07:27:46,720 adding 'colossalai/utils/profiler/stateful_tensor_mem_extention.py' 2023-03-10T07:27:46,723 adding 'colossalai/utils/profiler/legacy/__init__.py' 2023-03-10T07:27:46,727 adding 'colossalai/utils/profiler/legacy/comm_profiler.py' 2023-03-10T07:27:46,731 adding 'colossalai/utils/profiler/legacy/pcie_profiler.py' 2023-03-10T07:27:46,734 adding 'colossalai/utils/profiler/legacy/prof_utils.py' 2023-03-10T07:27:46,737 adding 'colossalai/utils/rank_recorder/__init__.py' 2023-03-10T07:27:46,741 adding 'colossalai/utils/rank_recorder/rank_recorder.py' 2023-03-10T07:27:46,744 adding 'colossalai/utils/tensor_detector/__init__.py' 2023-03-10T07:27:46,748 adding 'colossalai/utils/tensor_detector/tensor_detector.py' 2023-03-10T07:27:46,752 adding 'colossalai/zero/__init__.py' 2023-03-10T07:27:46,756 adding 'colossalai/zero/init_ctx/__init__.py' 2023-03-10T07:27:46,760 adding 'colossalai/zero/init_ctx/init_context.py' 2023-03-10T07:27:46,764 adding 'colossalai/zero/shard_utils/__init__.py' 2023-03-10T07:27:46,766 adding 'colossalai/zero/shard_utils/base_shard_strategy.py' 2023-03-10T07:27:46,769 adding 'colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py' 2023-03-10T07:27:46,771 adding 'colossalai/zero/shard_utils/commons.py' 2023-03-10T07:27:46,774 adding 'colossalai/zero/shard_utils/tensor_shard_strategy.py' 2023-03-10T07:27:46,778 adding 'colossalai/zero/sharded_model/__init__.py' 2023-03-10T07:27:46,781 adding 'colossalai/zero/sharded_model/_utils.py' 2023-03-10T07:27:46,785 adding 'colossalai/zero/sharded_model/reduce_scatter.py' 2023-03-10T07:27:46,793 adding 'colossalai/zero/sharded_model/sharded_model_v2.py' 2023-03-10T07:27:46,796 adding 'colossalai/zero/sharded_model/utils.py' 2023-03-10T07:27:46,800 adding 'colossalai/zero/sharded_optim/__init__.py' 2023-03-10T07:27:46,804 adding 'colossalai/zero/sharded_optim/_utils.py' 2023-03-10T07:27:46,812 adding 'colossalai/zero/sharded_optim/low_level_optim.py' 2023-03-10T07:27:46,818 adding 'colossalai/zero/sharded_optim/sharded_optim_v2.py' 2023-03-10T07:27:46,822 adding 'colossalai/zero/sharded_optim/bookkeeping/__init__.py' 2023-03-10T07:27:46,825 adding 'colossalai/zero/sharded_optim/bookkeeping/base_store.py' 2023-03-10T07:27:46,827 adding 'colossalai/zero/sharded_optim/bookkeeping/bucket_store.py' 2023-03-10T07:27:46,830 adding 'colossalai/zero/sharded_optim/bookkeeping/gradient_store.py' 2023-03-10T07:27:46,833 adding 'colossalai/zero/sharded_optim/bookkeeping/parameter_store.py' 2023-03-10T07:27:46,836 adding 'colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py' 2023-03-10T07:27:46,839 adding 'colossalai/zero/sharded_param/__init__.py' 2023-03-10T07:27:46,842 adding 'colossalai/zero/sharded_param/sharded_param.py' 2023-03-10T07:27:46,845 adding 'colossalai/zero/sharded_param/sharded_tensor.py' 2023-03-10T07:27:46,848 adding 'colossalai/zero/utils/__init__.py' 2023-03-10T07:27:46,851 adding 'colossalai/zero/utils/gemini_hook.py' 2023-03-10T07:27:46,854 adding 'colossalai/zero/utils/zero_hook.py' 2023-03-10T07:27:46,861 adding 'op_builder/__init__.py' 2023-03-10T07:27:46,864 adding 'op_builder/builder.py' 2023-03-10T07:27:46,867 adding 'op_builder/cpu_adam.py' 2023-03-10T07:27:46,870 adding 'op_builder/fused_optim.py' 2023-03-10T07:27:46,873 adding 'op_builder/layernorm.py' 2023-03-10T07:27:46,875 adding 'op_builder/moe.py' 2023-03-10T07:27:46,878 adding 'op_builder/multi_head_attn.py' 2023-03-10T07:27:46,881 adding 'op_builder/scaled_masked_softmax.py' 2023-03-10T07:27:46,883 adding 'op_builder/scaled_upper_triangle_masked_softmax.py' 2023-03-10T07:27:46,887 adding 'op_builder/utils.py' 2023-03-10T07:27:46,893 adding 'colossalai-0.2.7.dist-info/LICENSE' 2023-03-10T07:27:46,899 adding 'colossalai-0.2.7.dist-info/METADATA' 2023-03-10T07:27:46,901 adding 'colossalai-0.2.7.dist-info/WHEEL' 2023-03-10T07:27:46,903 adding 'colossalai-0.2.7.dist-info/entry_points.txt' 2023-03-10T07:27:46,905 adding 'colossalai-0.2.7.dist-info/top_level.txt' 2023-03-10T07:27:46,932 adding 'colossalai-0.2.7.dist-info/RECORD' 2023-03-10T07:27:46,982 removing build/bdist.linux-armv7l/wheel 2023-03-10T07:27:47,487 Building wheel for colossalai (setup.py): finished with status 'done' 2023-03-10T07:27:47,521 Created wheel for colossalai: filename=colossalai-0.2.7-py3-none-any.whl size=896479 sha256=d8c17d923035a789da65ce205a0daea196df83cec28f43390376dfca88775424 2023-03-10T07:27:47,523 Stored in directory: /tmp/pip-ephem-wheel-cache-_skaw1ct/wheels/2b/93/67/73311c342fc185a5c5df4cc9f287dfc0f5b5c38e03c148b322 2023-03-10T07:27:47,603 Successfully built colossalai 2023-03-10T07:27:47,692 Removed build tracker: '/tmp/pip-build-tracker-lxvge36r'